Remove all current uses of build-tools during buildworld

[arichardson]

Summary

I have always found it annoying that cd bin/sh && make clean && make all
does not work when cross-compiling. make clean removes the required host
tools and we then end up building them for the target system. This results
in a exec format error when we use them.

This commit passes down the host compiler to the buildworld/buildenv phases
to allow building host binaries during make all. This allows us to remove
lots of duplicate information from Makefile.inc1/Makefile.libcompat (since
they have to list all subdirs with build tools).

The build-tools phase still exists, but should be a no-op unless
LOCAL_TOOL_DIRS is defined. I will open a follow-up review to remove the
entire build-tools phase that can be committed once there are no more
users of LOCAL_TOOL_DIRS.

Obtained from: CheriBSD

Test Plan

amd64 buildworld still succeeds, and we have been using this in CheriBSD for a long time

[arichardson]

Rebase on top of D28181

[brooks]

I really like this. It's cleaner and eliminates a tree walk.

On weird use case this may impact is people doing a cross build and then mounting obj on a native host. If they do something that causes the bootstrap tools to be used they will be the wrong ABI. I think it's ok to break that, but who knows what sort of users will crawl out of the woodwork. It might be the case that we want a clean-bootstrap target to simplify rebuilding bootstrap tools.

bin/csh/Makefile

127

I do wonder if the ./ should be {.OBJDIR}/.

lib/libmagic/config.h

4

This feels like it should be a separate commit.

[sjg]

bin/csh/Makefile

127

Why change this? It was harmless as is, but potentially allows me to set BTOOLSPATH to a stage dir where I've put the tools build for pseudo machine "host" during DIRDEPS build.
More below

bin/sh/Makefile

56

At least use a variable. In our internal build we have:

`.if defined(HOST_OBJTOP)
MKNODES= ${HOST_OBJTOP}/${RELDIR}/mknodes/mknodes
MKSYNTAX= ${HOST_OBJTOP}/${RELDIR}/mksyntax/mksyntax

`
and use ${MKNODES} to build nodes.c and the default can be whatever you like.

But leaving the deafult as ${BTOOLSPATH:U.}/mknodes leaves open the option of building mknodes etc and staging them to say ${STAGE_HOST_OBJTOP}/bin and setting BTOOLSPATH to that.

This matters more when the target dir is built many. times and you want to only build the. host tools once...

[bdrewery]

Please test this with WITH_META_MODE=yes and kldload filemon. make buildworld -j and make buildworld -j -DNO_CLEAN. If any .meta: lines appear about these files then it's broken. A key point of build-tools was also using *different cflags*.

I commend the goal here but this will regress a lot of hard work.

Makefile.inc1

796–801

This is the wrong place to define BUILD_TOOLS_CC. How can I build directly in bin/sh now if BUILD_TOOLS_CC is not defined?

bin/csh/Makefile

127

Yeah this.

bin/sh/Makefile

49

This regresses ccache and .depend file usage fixes.

56

Yes we really need the BTOOLSPATH pattern to remain or be in the default PATH as the first entry.

[arichardson]

In D28253#632400, @bdrewery wrote:

Please test this with WITH_META_MODE=yes and kldload filemon. make buildworld -j and make buildworld -j -DNO_CLEAN. If any .meta: lines appear about these files then it's broken. A key point of build-tools was also using *different cflags*.

I commend the goal here but this will regress a lot of hard work.

I've never used META_MODE, and will have to find a system for testing this. It will probably be at least a week until I get around to it though.

Makefile.inc1

796–801

Is building without make buildenv supported? I always do cross-builds so I didn't consider that case.

bin/csh/Makefile

127

Will revert those changes.

bin/sh/Makefile

49

In what way does ccache regress? I don't use it so not sure what the expected behaviour is.
I just saw that most build tools used ${CC:N${CCACHE_BIN}} so I made COMPILER_ABSOLUTE_PATH strip ccache.

56

Will drop those changes and keep BTOOLSPATH.

share/syscons/scrnmaps/mkscrfil.c

32

Should probably be a separate commit.

[bdrewery]

Makefile.inc1

796–801

Yes

bin/sh/Makefile

49

git log --author=bdrewery on all Makefiles here will give some insight. It hasn't been easy to get all of the different build modes working together and keeping features like dependencies and caching working on top of that. Some of the problems go back decades.

A. For all of these except make -C bin/sh assume cross-build
B. For all of these they need both -B (-j0) and -jX tests

1. make buildworld
2. make buildworld -DNO_CLEAN # after a buildworld
3. make buildworld && make clean && make buildworld -DNO_CLEAN # this blows away .o files but keeps .depend files. It is almost the same as a fresh build but surprises happen here
4. make buildworld WITH_META_MODE=yes # this replaces -DNO_CLEAN by using bmake features to handle dependencies differently for incremental build. Not default mostly from lack of motivation/time and some lingering issues it does not yet fix that -DNO_CLEAN suffers from.
5. WITH_DIRDEPS_BUILD=yes make the-lot # this is a separate build system that primarily sjg has been maintaining that is still very promising and worth keeping up as it allows building things like bin/sh in the buildenv environment without much thought. Like you can just make -C bin/sh and it will Do The Right Thing using the proper buildenv. It has some rough edges still.
6. make -C bin/sh

Especially #6 but also #1-3 all need to work with existing and missing .depend.foo.o files, as well as with missing generated headers, updated source files.

And the install phases for each of these need to not be rebuilding anything. This is especially problematic for WITH_META_MODE.

[sjg]

The biggest "rough edge" with DIRDEPS_BUILD is lack of src dirs to represent packages or other natural targets in FreeBSD tree.
A directory is what the DIRDEPS_BUILD works with. In the Junos build and our internal FreeBSD build we have directories in which we build
each package (and each kernel we want), this makes build orchestration very simple.
Lack of it leaves you stuck with tree walks and special targets.

[arichardson]

Makefile.inc1

796–801

Looks like we now have HOST_CC with a default value so I'll have to rebase this at some point.