meta mode changes for existing makefiles other than share/mk

[sjg]

Summary

This diff shows changes to existing makefiles (other than share/mk/)

This is mostly for ease of folk to see and comment....

Most common change is removing NO_OBJ - for meta mode we want obj dirs
Some changes to deal with avoiding circular dependencies (we aim to build everything in a single pass - ie each dir visited only once)

Building clang is a lot of fun - a lot of noise in clang.build.mk to find tblgen that will work

The change to libpam moves the building of static modules to separate directory

Some changes can probably be revised - include/Makefile was done before stage-install.sh introduced

Test Plan

buildworld, as well as building most of userland, kernels etc in meta mode.

[imp]

Good first cut, but there's a lot of changes commingled here that should be busted up into separate commits.

Makefile

532 ↗

(On Diff #6006)

I'd set MK_META_MODE directly to 'no' here.

535 ↗

(On Diff #6006)

I'd set MK_STAGING=no here.

536 ↗

(On Diff #6006)

I'd be tempted to rename this to MK_UPDATE_DEPENDFILE. Is there a reason this doesn't follow the pattern?

kerberos5/lib/libasn1/Makefile

119 ↗

(On Diff #6006)

This is a weird one. (a) check to make sure it is really needed and (b) comment as to why it is really needed. There's only a few in the tree and that will help others to know whether to delete this line if they cut and paste from this file.

lib/clang/clang.build.mk

44 ↗

(On Diff #6006)

All the MACHINE == "host" things look very weird to me. Why is that needed?

lib/libc/Makefile

166 ↗

(On Diff #6006)

What's _SKIP_BUILD? Why's it needed?

lib/libpam/libpam/Makefile

175 ↗

(On Diff #6006)

Why delete openpam_stiatic? This doesn't make sense to me and isn't in the comments.

lib/libtelnet/Makefile

29 ↗

(On Diff #6006)

Why delete this? Doesn't seem related to the problem at hand.

lib/msun/Makefile

38 ↗

(On Diff #6006)

Doesn't seem related to the problem at hand. Perhaps this is 'always' right.

rescue/rescue/Makefile

226 ↗

(On Diff #6006)

This is a gratuitous change.

sbin/atm/atmconfig/Makefile

15 ↗

(On Diff #6006)

I don't understand this change.

share/monetdef/Makefile

141 ↗

(On Diff #6006)

All the symlink changes look good, but are a separate thing. Could be committed separately.

share/syscons/fonts/Makefile

35 ↗

(On Diff #6006)

remove the now-extra blank line.

share/syscons/keymaps/Makefile

55 ↗

(On Diff #6006)

Ditto. And in other places I've not flagged....

sys/conf/kmod.mk

349 ↗

(On Diff #6006)

This is a regression.

usr.bin/kdump/Makefile

36 ↗

(On Diff #6006)

The /usr/include -> ${INCLUDEDIR} seems like a separate change as well.

usr.bin/kdump/mkioctls

42 ↗

(On Diff #6006)

This looks good, but could also be a separate change.

usr.bin/mkcsmapper_static/Makefile

10 ↗

(On Diff #6006)

This looks like a separate change and could be done separately.

[brooks]

I'm not seeing anything significant that imp didn't comment on.

I'd like to see NO_OBJ, INCLUDEDIR, locale symlink fixes, etc broken out and committed separately. They seem to be things that are fine without meta_mode so should just go in to keep the meta_mode changes as small as possible

I'm not a fan of INSTALL_AS_USER's implementation or MTREE_FILTER. We should teach mtree to do the filtering (or just shut it up if that's what's required) and have install do the right thing. That said, we shouldn't hold up the meta_mode merge to fix mtree.

[sjg]

In D2748#52841, @brooks wrote:

I'm not seeing anything significant that imp didn't comment on.

Cool

I'd like to see NO_OBJ, INCLUDEDIR, locale symlink fixes, etc broken out and committed separately.

No problem - I was not intending this to represent one commit.
Simply capturing in one place the "interesting" changes that warrant discussion.

I'm not a fan of INSTALL_AS_USER's implementation or MTREE_FILTER. We should teach mtree to do the filtering (or just shut it up if that's what's required) and have install do the right thing. That said, we shouldn't hold up the meta_mode merge to fix mtree.

That would be fine with me.
It would be nice to be able to skip the mtree bit all together.
The staging logic is happy to make needed dirs as it goes.
Where we fail is when we need to run 'install' to put things in the stage tree and the makefile makes no provision for creating the dirs - assuming mtree did it.

Makefile

532 ↗

(On Diff #6006)

Yes, now that sys.mk sets these options, this can be cleaned up quite a bit.

536 ↗

(On Diff #6006)

Yes, UPDATE_DEPENDFILE belongs to meta.autodep.mk which is independent of freebsd etc. Also its setting is normally determined by circumstances.
The value NO (shouting), is special in that it tells meta.autodep.mk that we don't care what it thinks we don't want to update.

It may be possible to map this to the normal option model need to look.

lib/clang/clang.build.mk

44 ↗

(On Diff #6006)

"host" is a pseudo machine used when building tools for the build host.
It is special in many ways, since we do not use "host" in the objdir name, but rather $HOST_TARGET (eg freebsd10-amd64, netbsd5-i386, ..) which ensure that the same tree can be shared via NFS, built on multiple host types without confusion.

In something like a kernel makefile.depend you would expect a dependency on
usr.sbin/config.host

indicating that usr.sbin/config needs to be built for "host" so we can use it.
This helps avoid the need for separate pass of the tree to build "tools" and such.

In Junos we also make heavy use of the pseudo machine "common" for architecture independent generated files (junos generates a lot of the source it is built from), and using the pseudo machine allows that to be done once only, and the results re-used for the dozen or so architectures that need it.

lib/libc/Makefile

166 ↗

(On Diff #6006)

meta mode works best when all building happens at level 1+
level 0 being restricted to just orchestrating the build.
_SKIP_BUILD if non-empty is displayed as the reason why something isn'y being built.

eg.

not building at level 0
mips is not in supported machine list: amd64 arm i386

etc. In the projects/bmake branch so far we only use the level 0 one.
In junos it is used for several reasons like $MACHINE not in $ONLY_MACHINE_LIST
(to guard against humans accidentally asking a dir to be built for the wrong architecture).

lib/libpam/libpam/Makefile

175 ↗

(On Diff #6006)

It is moved to its own makefile so that it works better

lib/libtelnet/Makefile

29 ↗

(On Diff #6006)

It is a bug to have more than one makefile try to install the same header or library into the same destination.
The meta mode staging logic detects this and throws an error,
arpa/telnet.h is also installed by include/arpa

lib/msun/Makefile

38 ↗

(On Diff #6006)

It is always right, - without it we cannot break the circular dependency b/w this and libc

sbin/atm/atmconfig/Makefile

15 ↗

(On Diff #6006)

Based on the commit message

Qualifying targets with ${.OBJDIR}/ confuses make for little reason.

I'm going to guess that using .OBJDIR rather than . which is equivalent, resulted failure to recognize and capture the local dependencies on oid.h:

.if ${DEP_RELDIR} == ${_DEP_RELDIR}
\# local dependencies - needed for -jN in clean tree
atmconfig_device.o: oid.h
atmconfig_device.po: oid.h
.endif

it may no longer be needed - both bmake and the makefiles that post-process meta files have gotten smarter since that commit was done (2012).

share/syscons/keymaps/Makefile

55 ↗

(On Diff #6006)

Will do

sys/conf/kmod.mk

349 ↗

(On Diff #6006)

Likely a merge botch on my part

[imp]

I'd be tempted to kill INSTALL_AS_USER entirely. It's useless today with the meta file stuff. It doesn't add any value. It fails in a number of ways because people don't test it and break it w/o realizing it. I know this affects Juniper, and I don't care. Well, it isn't that I don't care, it's that I don't think that should hold FreeBSD back that Juniper hasn't migrated yet. Then again, I get confused between meta mode and the meta file stuff...

[sjg]

reflect some review comments