Simplify building host tools for non-FreeBSD hosts

[sjg]

Summary

Fix tools/build to stage the headers needed to be able to build
tools like makefs, mkimg and bsdtar for hosts like Linux.

If using DIRDEPS_BUILD, MACHINE=host make -j8 host-tools
should be all that is needed.

On Linux the libsbuf-dev package is needed for makefs, and
tar needs libbsd as well.

Sponsored by: Juniper Networks, Inc.

[emaste]

contrib/libarchive/tar/bsdtar.c

206 ↗

(On Diff #120237)

If necessary this one should be done through https://github.com/libarchive/libarchive

However it looks like the proper fix is to have HAVE_D_MD_ORDER be defined in your cross-build env, from upstream:

CMakeLists.txt:CHECK_SYMBOL_EXISTS(D_MD_ORDER       "langinfo.h" HAVE_D_MD_ORDER)

tools/build/cross-build/include/README

6 ↗

(On Diff #120237)

?

[sjg]

Avoid touching contrib/libarchive

[sjg]

contrib/libarchive/tar/bsdtar.c

206 ↗

(On Diff #120237)

The problem is, lib/libarchive/config_freebsd.h defines HAVE_D_MD_ORDER
which is superfluous considering D_MD_ORDER is a define.
I can modify lib/libarchive/config_freebsd.h to use #ifndef which would avoid touching code from upstream.

tools/build/cross-build/include/README

6 ↗

(On Diff #120237)

Its from obrien

[obrien]

lib/libarchive/config_freebsd.h

103–105 ↗

(On Diff #120240)

This file is for targeting FreeBSD, no? Seems you should be creating a config_linux.h or config_ubuntu.h. And then add a case for that in lib/libarchive/Makefile where we currently do
Makefile.inc:CFLAGS+= -DPLATFORM_CONFIG_H=\"${.CURDIR}/config_freebsd.h\"

share/mk/src.init.linux.mk

19

Any idea if this path is at all standardized across Linuxes?

share/mk/src.init.mk

13–15

Are you ending an .if, to reopen it with the same logic for stylistic reasons? Would it be nicer as one .if block?

16

Maybe I'm misreading this, but Should you use HOST_INCLUDE_ROOT, which already has logic to define appropriately.

targets/pseudo/host-tools/Makefile.depend.host

3–5

Sort.

tools/build/cross-build/include/README

3–4 ↗

(On Diff #120240)

My latest text is:
+1. The FreeBSD sys/sys/_types.h should not be included
+ Any <sys/_types.h> on the native build host should be used instead.
+ <sys/types.h> needs to be the host's - the types must match the host
+ kernel and host libc.
+
+2. tools/build/cross-build/include/common/ headers should be included
+ before tools/build/cross-build/include/<platform>/ headers where
+ the same header name exists.

tools/build/cross-build/include/common/grp.h

52–78

Why is there #ifdef linux anywhere in a common header instead of in tools/build/cross-build/include/linux/grp.h?

tools/build/cross-build/include/linux/sys/types.h

42–43 ↗

(On Diff #120240)

I'm still puzzled why you're not getting <sys/sysmacros.h> included from tools/build/cross-build/include/common/sys/types.h :
#if __has_include(<sys/sysmacros.h>)
/* GLibc defines makedev/minor/major in sysmacros.h instead of sys/types.h */
#include <sys/sysmacros.h>
#endif

Though we know glibc isn't used on macOS and only Linux. So I don't know why those lines are in a common header to begin with.

But that doesn't answer the question on what's going on. Are you getting both tools/build/cross-build/include/common/sys/types.h and tools/build/cross-build/include/linux/sys/types.h included?

usr.sbin/makefs/mtree.c

1060–1065 ↗

(On Diff #120240)

Why do we need the #if HAVE_ tests? macOS has setgroupent()/setpassent() natively. Linux has it thru tools/build/cross-build/include/common/grp.h above (which really should be moved out of the common header since macOS has it natively).

[sjg]

Add config_linux.h for libarchive

[sjg]

share/mk/src.init.linux.mk

19

Sorry, I have no idea. If/when someone wants to adapt to other distros we will find out.

[sjg]

Only use STAGE_INCLUDEDIR if staging

[brooks]

This should be pushed to GitHub and tested as a pull request before commit. That will ensure bootstrapping various bits works across the environments we test.

[jrtc27]

This seems to be unfinished, a bit of a mess and doing way too many things at once to have any hope of properly reviewing it

share/mk/src.init.linux.mk

19

There are architectures other than x86 that we support...

[sjg]

In D39541#900496, @brooks wrote:

This should be pushed to GitHub and tested as a pull request before commit. That will ensure bootstrapping various bits works across the environments we test.

Sorry, what is the process for doing that?

share/mk/src.init.linux.mk

19

For building on?
Do you have a list?
Do you happen to know what valid ld-linux-* path is?

[jrtc27]

share/mk/src.init.linux.mk

19

I believe PowerPC and AArch64 have both been made to work. Why can't you leave it up to the host toolchain? The names are a giant inconsistent mess over in Linux land.

[jrtc27]

In D39541#900502, @sjg wrote:

In D39541#900496, @brooks wrote:

This should be pushed to GitHub and tested as a pull request before commit. That will ensure bootstrapping various bits works across the environments we test.

Sorry, what is the process for doing that?

Just open a PR against freebsd/freebsd-src

[sjg]

Allow for non-x86 host

[sjg]

Leave rtld to toolchain

[sjg]

share/mk/src.init.linux.mk

19

Could be a bug in the toolchain we are using - I can move this to a site.init

share/mk/src.init.mk

16

This is the arg to -isystem in local.init.mk it does not necessarily have anything to do with host though this value does
Or do you mean ISYSTEM= ${HOST_INCLUDE_ROOT} ?

tools/build/cross-build/include/README

3–4 ↗

(On Diff #120240)

I can drop this from here so you can commit as you like?

tools/build/cross-build/include/common/grp.h

52–78

Good question

tools/build/cross-build/include/linux/sys/types.h

42–43 ↗

(On Diff #120240)

This is left over - removing

[sjg]

Cleanup src.init.mk

[sjg]

Fix logic fail in libc

[sjg]

FWIW buildworld on FreeBSD 13.1 completes ok

[sjg]

ISYSTEM is only relevant when MK_STAGING is yes

[sjg]

Need pseudo option for building on older or non-FreeBSD hosts
so we build libegacy (tools/build)