Add option UNIFIED_OBJDIR, on by default, which moves the default build OBJDIR.

[bdrewery]

Summary

This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>. This
new format is used regardless of cross or native build.

The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.

Relnotes: yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work: D3711 D874
Sponsored by: Dell EMC Isilon

The UNIFIED_OBJDIR option is intended to be a temporary migration option.

Uses and expands some of the DIRDEPS_BUILD logic, moved in rS325191.

AUTO_OBJ is _not enabled_ in this commit, D12841 handles it.

Test Plan

Ran all of buildworld, installworld, buildkernel, installkernel, DIRDEPS_BUILD, native-xtools, native-xtools-install, xdev, xdev-links, cleanworld.

Still need to do another universe pass but I expect no problems after fixing
a few unrelated things like rS325180.

[bdrewery]

share/mk/src.sys.obj.mk

96–101 ↗

(On Diff #34474)

FYI @sjg has already fixed this in NetBSD bmake but I am keeping here for now. It should cause no harm.

[bdrewery]

Rebase

[bdrewery]

share/mk/src.sys.obj.mk

66 ↗

(On Diff #34479)

The DIRDEPS_BUILD build has a nice feature of a TARGET_SPEC for defining how the obdir will look. For now I am sticking to the simple TARGET.TARGET_ARCH and may expand on this later.

[gjb]

It looks like lines 19-20 of release/tools/vmimage.subr should also be updated so the path to the build pmbr is properly located.

[bdrewery]

tools/tools/build_option_survey/option_survey.sh

12–21 ↗

(On Diff #34479)

None of this change is needed here it seems. Testing.

[bdrewery]

• Fix some release/ scripts to use sys/boot .OBJDIR.
• Revert option_survey change as it really doesn't care where the files are.

[sjg]

This generally looks like a good improvement

share/mk/local.meta.sys.mk

10 ↗

(On Diff #34569)

comment should be removed also

share/mk/src.sys.obj.mk

9 ↗

(On Diff #34569)

Worth noting that make will ignore MAKEOBJDIR if MAKEOBJDIRPREFIX is used.

49 ↗

(On Diff #34569)

# see http://www.crufty.net/sjg/docs/sb-tools.htm

101 ↗

(On Diff #34569)

The correct .OBJDIR for ${SRCTOP} would be ${OBJTOP}

[bdrewery]

share/mk/src.sys.obj.mk

101 ↗

(On Diff #34569)

That assumes MK_AUTO_OBJ is on which in this block it is disabled. Note _MAKEOBJDIRPREFIX_ and _MAKEOBJDIR_ have an exists check first.
It's debatable if _.OBJDIR_ should be _OBJTOP_ here... I do like having a proper _.OBJDIR_ to evaluate in the top-level Makefiles even before the objdir exists, but in that case _OBJTOP_ can be used or better yet we can rely on _MK_AUTO_OBJ_ being enabled by default; this gives the same .OBJDIR as normal when the top-level does not have an obj dir created.

This is just working around the bug you fixed via email this week. I will add && ${MAKE_VERSION} <= 20170720 to only cover bugged versions.

[bdrewery]

Review findings

[emaste]

Should we make a reference to WITHOUT_UNIFIED_OBJDIR being a transition aid that will be removed later on? (for 12.0)?

share/mk/src.sys.obj.mk

29 ↗

(On Diff #34572)

maybe just "historical behaviour"?

I don't think the unified objdir is more complicated, at least for anyone who builds multiple architectures.

tools/build/options/WITHOUT_UNIFIED_OBJDIR

2 ↗

(On Diff #34572)

not use the newer style?
use the older style object directory?

[sjg]

In Junos build we have always used .OBJDIR = ${OBJTOP} for the case ${.CURDIR} == ${SRCTOP} which is why I asserted that it is the correct answer ;-)

[gjb]

share/mk/src.sys.obj.mk

107 ↗

(On Diff #34572)

It looks like there is a missing '.endif' here. With this applied, I see:

root@:/usr/src # make -s buildworld buildkernel
make: "/usr/src/share/mk/src.sys.obj.mk" line 0: 1 open conditional
make: Fatal errors encountered -- cannot continue

[bdrewery]

In D12840#267376, @sjg wrote:

In Junos build we have always used .OBJDIR = ${OBJTOP} for the case ${.CURDIR} == ${SRCTOP} which is why I asserted that it is the correct answer ;-)

How about I do it in a separate commit. This commit doesn't handle MK_AUTO_OBJ, so it is just setting .OBJDIR if possible to an existing directory. I am keeping the current behavior of .OBJDIR==.CURDIR when there's no OBJDIR yet in SRCTOP. I do prefer having a valid .OBJDIR to use even if 'make obj' hasn't ran yet. So I am open to doing it, I just think it's a behavior change that deserves its own commit.

[bdrewery]

share/mk/src.sys.obj.mk

107 ↗

(On Diff #34572)

Thanks for testing. Seems it was lost with a recent rebase, will post an update in a few minutes.

[bdrewery]

In D12840#267370, @emaste wrote:

Should we make a reference to WITHOUT_UNIFIED_OBJDIR being a transition aid that will be removed later on? (for 12.0)?

My UPDATING comment was intended for that but rereading it I see it is not clear. I will reword it and the manpage entry as well.

[bdrewery]

• Fix .if to be a proper .elif
• Clarify UPDATING and man pages on option
• Other review findings

[bdrewery]

Fix WITHOUT_UNIFIED_OBJDIR to actually work and only apply it at top-level like was done historically.

[bdrewery]

share/mk/src.sys.obj.mk

107 ↗

(On Diff #34572)

It turned out that the .if needed to be .elif.

[bdrewery]

In D12840#267602, @bdrewery wrote:

In D12840#267376, @sjg wrote:

In Junos build we have always used .OBJDIR = ${OBJTOP} for the case ${.CURDIR} == ${SRCTOP} which is why I asserted that it is the correct answer ;-)

How about I do it in a separate commit. This commit doesn't handle MK_AUTO_OBJ, so it is just setting .OBJDIR if possible to an existing directory. I am keeping the current behavior of .OBJDIR==.CURDIR when there's no OBJDIR yet in SRCTOP. I do prefer having a valid .OBJDIR to use even if 'make obj' hasn't ran yet. So I am open to doing it, I just think it's a behavior change that deserves its own commit.

Actually it can't even work unless the directory exists, which is all already handled above this block and later by auto.obj.mk.

~/git/freebsd2 # MAKEOBJDIRPREFIX=/nonexistent WITHOUT_AUTO_OBJ=yes make -V .OBJDIR -V OBJTOP
make: "/root/git/freebsd2/share/mk/src.sys.obj.mk" line 162: .OBJDIR: /nonexistent/root/git/freebsd2/amd64.amd64
/root/git/freebsd2
/nonexistent/root/git/freebsd2/amd64.amd64

[sjg]

In D12840#267602, @bdrewery wrote:

In D12840#267376, @sjg wrote:

In Junos build we have always used .OBJDIR = ${OBJTOP} for the case ${.CURDIR} == ${SRCTOP} which is why I asserted that it is the correct answer ;-)

How about I do it in a separate commit. This commit doesn't handle MK_AUTO_OBJ, so it is just setting .OBJDIR if possible to an existing directory. I am keeping the current behavior of .OBJDIR==.CURDIR when there's no OBJDIR yet in SRCTOP. I do prefer having a valid .OBJDIR to use even if 'make obj' hasn't ran yet. So I am open to doing it, I just think it's a behavior change that deserves its own commit.

That's fine by me.