Build work 11.0 plans status update

[Bryan Drewery]

(bcc'd some specific interested parties)

This is from a private mail I sent to re@ a while back and is a status
update for upcoming work.

Heads up, I intend to continue adding a few new features during the
slush/stable period since they are so impactful. They will be
off-by-default for stable/11 at this point I guess. I feel this is fine
since it is not ABI-related.
- AUTO_OBJ: For subdir builds and buildworld, automatically create obj
dirs without needing 'make obj'. I wanted to enable this by default but
can wait for the branch to be created. There is some work pending for this.
- Using objdir pattern of /usr/obj/usr/src/TARGET.TARGET_ARCH/bin/sh for
all archs, not just non-native. I'd prefer this made it into stable/11.
Very little work remaining for this.
- WITH_META_MODE: Fixed incremental (with default -DNO_CLEAN) build
commit. Not planning on-by-default for stable/11.
- Build clang once for universe regardless of WITH_SYSTEM_COMPILER.
On-by-default. I really intend to make this into stable/11. It will be
too great of a time savings to ignore for 11 release testing IMO.

Just an update...

- WITH_SYSTEM_COMPILER is now on by default. There has been some
unexpected fallout from this such as with ppc64 thinking it was using
a newer gcc (fixed), or the xlint build failing for 1 user (still not
fixed).

- META_MODE should mostly be done now. Some fixes/tweaks may still
trickle in for edge cases, such as allowing WITH_SYSTEM_COMPILER to work
with it. The problem is that flip-flopping between using an internal
and external compiler and it adding -target/--sysroot flags in on the
2nd build and meta mode detecting the changed build command. I would
like to always pass these flags, even with the internal cross-compiler,
to fix the problem. I think the lld effort would be happy with this too.

- Building clang once in universe isn't coming soon. Likely 3-4 weeks
out. I'll cross the question of merging once it is done and how
intrusive it is. I believe it may be a bit more complex than I imagined
to make it clean and easily maintained. It would need a pre-universe
"universe-toolchain" phase that changes how universe works a lot. I
consider it overall trivial but don't want to duplicate too much code.
It somewhat relies on the new AUTO_OBJ work too. More time is needed on it.

- AUTO_OBJ I am splitting into 3 separate patches after feedback.
1. Removing 'make obj' treewalks from buildworld/buildkernel. This is
coming in the next few days. No risk to it.
2. Unifying the objtree as like
/usr/obj/usr/src/TARGET.TARGET_ARCH/bin/sh. This I will bring in
an option of UNIFIED_OBJDIR to flip between the old and new
patterns. Some tools such as picobsd and the options survey will
rely on the old pattern. A CFT is needed to discover what else
relies on the old patterns still. There's also a problem of
migrating from the old tree to the new. I will bring this patch
as off-by-default to re@ in the next week.
3. Default AUTO_OBJ in subdirs. This one is kind of tricky and needs
some more thought. Non-root especially may be problematic since
it will try to write to /usr/obj by default. I likely will hold
off on this and not propose it for 11.0.

[Glen Barber]

Post by Bryan Drewery
(bcc'd some specific interested parties)
update for upcoming work.

Heads up, I intend to continue adding a few new features during the
slush/stable period since they are so impactful. They will be
off-by-default for stable/11 at this point I guess. I feel this is fine
since it is not ABI-related.
- AUTO_OBJ: For subdir builds and buildworld, automatically create obj
dirs without needing 'make obj'. I wanted to enable this by default but
can wait for the branch to be created. There is some work pending for this.
- Using objdir pattern of /usr/obj/usr/src/TARGET.TARGET_ARCH/bin/sh for
all archs, not just non-native. I'd prefer this made it into stable/11.
Very little work remaining for this.
- WITH_META_MODE: Fixed incremental (with default -DNO_CLEAN) build
commit. Not planning on-by-default for stable/11.
- Build clang once for universe regardless of WITH_SYSTEM_COMPILER.
On-by-default. I really intend to make this into stable/11. It will be
too great of a time savings to ignore for 11 release testing IMO.

Just an update...
- WITH_SYSTEM_COMPILER is now on by default. There has been some
unexpected fallout from this such as with ppc64 thinking it was using
a newer gcc (fixed), or the xlint build failing for 1 user (still not
fixed).
- META_MODE should mostly be done now. Some fixes/tweaks may still
trickle in for edge cases, such as allowing WITH_SYSTEM_COMPILER to work
with it. The problem is that flip-flopping between using an internal
and external compiler and it adding -target/--sysroot flags in on the
2nd build and meta mode detecting the changed build command. I would
like to always pass these flags, even with the internal cross-compiler,
to fix the problem. I think the lld effort would be happy with this too.
- Building clang once in universe isn't coming soon. Likely 3-4 weeks
out. I'll cross the question of merging once it is done and how
intrusive it is. I believe it may be a bit more complex than I imagined
to make it clean and easily maintained. It would need a pre-universe
"universe-toolchain" phase that changes how universe works a lot. I
consider it overall trivial but don't want to duplicate too much code.
It somewhat relies on the new AUTO_OBJ work too. More time is needed on it.
- AUTO_OBJ I am splitting into 3 separate patches after feedback.
1. Removing 'make obj' treewalks from buildworld/buildkernel. This is
coming in the next few days. No risk to it.
2. Unifying the objtree as like
/usr/obj/usr/src/TARGET.TARGET_ARCH/bin/sh. This I will bring in
an option of UNIFIED_OBJDIR to flip between the old and new
patterns. Some tools such as picobsd and the options survey will
rely on the old pattern. A CFT is needed to discover what else
relies on the old patterns still. There's also a problem of
migrating from the old tree to the new. I will bring this patch
3. Default AUTO_OBJ in subdirs. This one is kind of tricky and needs
some more thought. Non-root especially may be problematic since
it will try to write to /usr/obj by default. I likely will hold
off on this and not propose it for 11.0.

Regardless of if some of this does not make it into 11.0-RELEASE, thank
you very much for all your hard work on these fixes, features, updates,
and bug fixes.

Glen

[Simon J. Gerraty]

Post by Bryan Drewery
3. Default AUTO_OBJ in subdirs. This one is kind of tricky and needs
some more thought. Non-root especially may be problematic since
it will try to write to /usr/obj by default. I likely will hold
off on this and not propose it for 11.0.

Surely this can be made dependent on writability of OBJROOT (/usr/obj/)?

[Bryan Drewery]

Post by Simon J. Gerraty

Post by Bryan Drewery
3. Default AUTO_OBJ in subdirs. This one is kind of tricky and needs
some more thought. Non-root especially may be problematic since
it will try to write to /usr/obj by default. I likely will hold
off on this and not propose it for 11.0.

Surely this can be made dependent on writability of OBJROOT (/usr/obj/)?

Maybe. I'd rather not introduce something like this since it is invoked
from sys.mk. I really hate adding more executions. To bad there is no
make writable() function...

<MAKEOBJDIR / OBJROOT / OBJTOP determined here>

CAN_WRITE!= test -w ${MAKEOBJDIR} && echo 1 || echo 0
# The above test is not really enough either since we want to test every
component of the directory which is what mkdir -p is already doing in
auto.obj.mk...
.if ${CAN_WRITE} == 1
__DEFAULT_YES_OPTIONS+= AUTO_OBJ
.endif

.include <bsd.mkopt.mk>

.if ${MK_AUTO_OBJ} == "yes"
.include <auto.obj.mk>
.endif


What seems more efficient to me is to default AUTO_OBJ to on and then
have auto.obj.mk consider an EPERM to just disable the feature for some
cases. The 'for some cases' is where it becomes murky since we want to
force auto obj on for things like buildworld/DIRDEPS regardless of
writability as those should error and stop. Building a user or in a
subdir (not DIRDEPS) may be fine to ignore the EPERM and build in source
tree since that's a historical behavior.

[Simon J. Gerraty]

Post by Bryan Drewery
Maybe. I'd rather not introduce something like this since it is invoked
from sys.mk. I really hate adding more executions. To bad there is no

Yes but you need only do it once - ie. if .MAKE.LEVEL==0
the result isn't going to change.

.if ${.MAKE.LEVEL} == 0
x != test -w ${OBJROOT:U${MAKEOBJDIRPREFIX:U/usr/obj}}/.; echo $$?
if ${x} != 0 && defined(WITH_AUTO_OBJ)
# they explicitly asked for it and we cannot do it
.error cannot write ${OBJROOT:U${MAKEOBJDIRPREFIX:U/usr/obj}} fix that or set WITHOUT_AUTO_OBJ
.endif
.endif

Post by Bryan Drewery
# The above test is not really enough either since we want to test every
component of the directory which is what mkdir -p is already doing in
auto.obj.mk...

If you are allowed to write /usr/obj/.
you should be able to write anything below it.

Of course using a OBJROOT private to the tree, avoids such problems.

SRCTOP=/some/where/src
OBJROOT=/some/where/obj

which is what we use....

Post by Bryan Drewery
What seems more efficient to me is to default AUTO_OBJ to on and then
have auto.obj.mk consider an EPERM to just disable the feature for some
cases. The 'for some cases' is where it becomes murky since we want to
force auto obj on for things like buildworld/DIRDEPS regardless of
writability as those should error and stop. Building a user or in a
subdir (not DIRDEPS) may be fine to ignore the EPERM and build in source
tree since that's a historical behavior.

auto.obj.mk will already throw an error if it cannot do what it was
asked to.

I like brutal simplicity.

That is much safer than soldiering on dribbling all over src for an hour
when that is not what the user wanted.

[John Baldwin]

Post by Bryan Drewery
(bcc'd some specific interested parties)
update for upcoming work.

Heads up, I intend to continue adding a few new features during the
slush/stable period since they are so impactful. They will be
off-by-default for stable/11 at this point I guess. I feel this is fine
since it is not ABI-related.
- AUTO_OBJ: For subdir builds and buildworld, automatically create obj
dirs without needing 'make obj'. I wanted to enable this by default but
can wait for the branch to be created. There is some work pending for this.

Will there be a way to disable use of /usr/obj if desired?

Normally I do want it, but sometimes I don't. One of my use cases is when
I have a src tree mounted via NFS into a VM guest and /usr/obj is private
to the guest. Being able to build "in-tree" in a work checkout means that
the binary is available on the host so I can run a debugger against it,
and/or I can build the binary in one place and run it in both.

For my work with gdb which uses auto*, I use a 'obj' subdir of the checkout
which is akin to what Simon suggests, but I can do that on a per-tree basis
without having to set various env vars or having to specify make vars on each
make invocation.

Also, when building random little source files ('vi foo.c' / 'make foo') it's
handy to be able to ./foo instead of /usr/obj/<tab><tab><tab>/foo to find
the binary I just built.

[Bryan Drewery]

Post by John Baldwin

Post by Bryan Drewery
(bcc'd some specific interested parties)
update for upcoming work.

Heads up, I intend to continue adding a few new features during the
slush/stable period since they are so impactful. They will be
off-by-default for stable/11 at this point I guess. I feel this is fine
since it is not ABI-related.
- AUTO_OBJ: For subdir builds and buildworld, automatically create obj
dirs without needing 'make obj'. I wanted to enable this by default but
can wait for the branch to be created. There is some work pending for this.

Will there be a way to disable use of /usr/obj if desired?
Normally I do want it, but sometimes I don't. One of my use cases is when
I have a src tree mounted via NFS into a VM guest and /usr/obj is private
to the guest. Being able to build "in-tree" in a work checkout means that
the binary is available on the host so I can run a debugger against it,
and/or I can build the binary in one place and run it in both.
For my work with gdb which uses auto*, I use a 'obj' subdir of the checkout
which is akin to what Simon suggests, but I can do that on a per-tree basis
without having to set various env vars or having to specify make vars on each
make invocation.
Also, when building random little source files ('vi foo.c' / 'make foo') it's
handy to be able to ./foo instead of /usr/obj/<tab><tab><tab>/foo to find
the binary I just built.

Yes, the context you quoted was an earlier description rather than the

Post by John Baldwin
- AUTO_OBJ I am splitting into 3 separate patches after feedback.
1. Removing 'make obj' treewalks from buildworld/buildkernel. This is
coming in the next few days. No risk to it.
2. Unifying the objtree as like
/usr/obj/usr/src/TARGET.TARGET_ARCH/bin/sh. This I will bring in
an option of UNIFIED_OBJDIR to flip between the old and new
patterns. Some tools such as picobsd and the options survey will
rely on the old pattern. A CFT is needed to discover what else
relies on the old patterns still. There's also a problem of
migrating from the old tree to the new. I will bring this patch
3. Default AUTO_OBJ in subdirs. This one is kind of tricky and needs
some more thought. Non-root especially may be problematic since
it will try to write to /usr/obj by default. I likely will hold
off on this and not propose it for 11.0.

This last bit (3) is what you're asking about and is why I am more
hesitant to push it forward too quickly. People often do want to build
in the local directory. You would be able to disable it with
'WITHOUT_AUTO_OBJ=yes make', if it were on by default.

The argument for enabling this in a subdir build by default is because
it is too easy to get into a situation where stale files in the source
tree break the build and lead to confusion. I get reports of this at
least once a week lately. I don't want to cause more problems by
enabling it though.