Thanks for upstreaming this! I think there are still a few files that installsysroot installs that aren't really needed (NLS?) but I needs to check whether that's still the case.

Rebased on latest HEAD.

The package also creates a tests user. Not sure if that is still required.

D24091, D24092 and D24093 are needed to build all universe kernels.
I have only tried booting MIPS n64 MALTA since I don't have a setup for o32/n32.

Rebase after LLVM 10 import

I'm currently on holiday but will update this once I'm back on Monday.

In D17002#526349, @dim wrote:

In D17002#506769, @arichardson wrote:

Ping? Can we merge this or do I have to convince upstream LLVM to add checks for the FreeBSD _FOO_DECLARED macros to their stddef.h/stdarg.h? I'm not sure how willing they would be to make that change though.

This should be solved upstream, once and for all. There is a reason we don't install those headers, the port shouldn't install them either.

I think they should be installed. It allows e.g. building binaries for a bare metal target. Fixing it in upstream llvm would be ideal but we could also change the FreeBSD headers to check for the llvm macros?

How does this compare to using the __atomic compiler builtins?

In D23534#516526, @kevans wrote:

In D23534#516493, @arichardson wrote:

For crossbuilding: It seems like macos ln doesn't support ln -r so we should avoid making future changes that use ln -r in build scripts. I guess otherwise we could bootstrap ln on macos, which should also work.

Hmm...certctl may be ran as a standard part of mergemaster/etcupdate. Do we assume these have a chance of working on, e.g., MacOS?

Change looks fine to me.

Allow overriding the LMA as may be required for some targets (requested by @jrtc27)

I can commit this now if we are okay with octeon kernels being broken until llvm 10 lands?

Yes I agree, removing the LTO option is not necessary.
I think in https://github.com/CTSRD-CHERI/cheribsd/commit/8041a7e459d1d6e453d082ff57db2f3f94ca1a3c I just copied from the genassym.o line.

In D23229#509463, @jhb wrote:

Yes, I was testing this locally, though without the :=, just a plain =. I was testing it as part of a make tinderbox run though that is only on amd64 and hasn't gotten to mips, so hadn't seen breakage that would require :=.

In D23204#509367, @jhb wrote:

In D23204#509143, @arichardson wrote:

mips64hf is currently broken because libc_nossp is built as soft-float (I think this is the lib32 version only since the error happens when building rtld-elf32):

ld: error: /home/alr48/obj/build/freebsd-universe-build/exports/users/alr48/sources/freebsd-universe/mips.mips64hf/libexec/rtld-elf32/rtld_libc.a(lstat.nossppico): floating point ABI '-msoft-float' is incompatible with target floa
ting point ABI '-mdouble-float'

I almost fixed this when fixing the mips64el case. Probably what we should do is set the 32-bit MACHINE_ARCH to MACHINE_ARCH:s/64// or some such to preserve el and hf suffixes.
However, note that part of the real problem here is that rtld-elf32 is somewhat broken in that it lets CFLAGS from the non-compat MACHINE_ARCH "leak" through. This doesn't
happen for the lib32 stage which is why the lib32 stage was building as soft-float since the MACHINE was "mips" but does happen for rtld-elf32. That's probably a bug in the bsd.compat.mk thing that @brooks might want to look at.

In D23204#509149, @arichardson wrote:

I also tried building n32 but that fails when linking the syscall objects. My guess is that SYS.h is missing the PIC n32 case and always generates position dependent assembly.

ld: error: can't create dynamic relocation R_MIPS_HI16 against symbol: __cerror in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in cerror.pico
>>> referenced by shm_rename.S:4
>>>               shm_rename.pico:(__sys_shm_rename)

I also tried building n32 but that fails when linking the syscall objects. My guess is that SYS.h is missing the PIC n32 case and always generates position dependent assembly.

mips64hf is currently broken because libc_nossp is built as soft-float (I think this is the lib32 version only since the error happens when building rtld-elf32):

Looks fine to me but someone else should give the final approval.

I guess I could try to build and boot a n32 world, but I'm not sure which kernel I should be using to test it.

No longer accurate, I have changes to allow bootstrapping localedef and CDDL code.

I am not sure what the correct process is to merge upstream LLVM changes. Is it fine to just commit this diff or are any other steps required?

Ping? Can we merge this or do I have to convince upstream LLVM to add checks for the FreeBSD _FOO_DECLARED macros to their stddef.h/stdarg.h? I'm not sure how willing they would be to make that change though.

I agree that a small separate tool would be nicer. I don't really have time to write one right now though. I'll wait before committing this until it's the last blocker for cross-building. Maybe by then I'll find some time to add a write_locked (or whatever name we want to choose) bootstrap tool.

Ping?

Looks good. Compiles for me with latest LLVM.

In D22630#497926, @cem wrote:

This is for cross-building?
FreeBSD's stddef.h includes sys/cdefs.h. If you aren't finding FreeBSD's stddef.h in a cross build, why would you expect to find FreeBSD's sys/cdefs.h?
I guess glibc's sys/cdefs.h includes __BEGIN_DECLS, etc, but I don't know how portable that assumption is (no idea what MacOS has, and I don't know if windows environments like mingw64 have a sys/cdefs.h).
I'd prefer a more holistic approach — use the src tree FreeBSD's headers when we're cross-compiling, rather than the host's.

In D22464#491516, @kevans wrote:

In D22464#491447, @arichardson wrote:

In D22464#491412, @kevans wrote:

In D22464#491409, @arichardson wrote:

LGTM. Only concern might be that adding /usr/libexec to PATH adds too much stuff.
Hopefully I can get the strict tmppath usable by default soon so it shouldn't matter anymore then.

Yeah, so what I really wanted here with flua was some way to indicate that flua can/should be symlinked in from the host (preferably to the same location that I've added to BPATH) on the versions it's *not* being bootstrapped (__FreeBSD_version after I added it) to simplify this, since I don't really want to add /usr/libexec. Any advice on making that happen (or just getting flua into one of the standard BPATH/XPATH paths while leaving it installing to /usr/libexec) is quite welcome. =)

It should be possible to modify tools/build/Makefile and change the host-symlinks target to add a flua symlink if it exists on the host.
Something like (untested):

.if exists(/usr/libexec/flua)
    mkdir -p ${DESTDIR}/usr/libexec
    ln -sf /usr/libexec/flua ${DESTDIR}/usr/libexec/flua
.endif

Yeah, I like that, thanks! I think the mkdir -p can safely be omitted -- we create it in the installdirs target now in case it's bootstrapped, which should always be run prior to host-symlinks if I understand correctly. If we later determine flua needs to be bootstrapped, that just gets built+installed over the symlink, right?

In D22464#491412, @kevans wrote:

In D22464#491409, @arichardson wrote:

LGTM. Only concern might be that adding /usr/libexec to PATH adds too much stuff.
Hopefully I can get the strict tmppath usable by default soon so it shouldn't matter anymore then.

Yeah, so what I really wanted here with flua was some way to indicate that flua can/should be symlinked in from the host (preferably to the same location that I've added to BPATH) on the versions it's *not* being bootstrapped (__FreeBSD_version after I added it) to simplify this, since I don't really want to add /usr/libexec. Any advice on making that happen (or just getting flua into one of the standard BPATH/XPATH paths while leaving it installing to /usr/libexec) is quite welcome. =)

LGTM. Only concern might be that adding /usr/libexec to PATH adds too much stuff.

In D17002#489675, @jhb wrote:

Does this work with GCC? Seems no:

> mips-unknown-freebsd12.1-gcc -print-resource-dir
mips-unknown-freebsd12.1-gcc: error: unrecognized command line option '-print-resource-dir'; did you mean '-print-search-dirs'?
mips-unknown-freebsd12.1-gcc: fatal error: no input files
compilation terminated.

This change is required to build with an unmodified upstream LLVM since the default upstream clang builtin headers are incompatible with the ones in /usr/bin/include (see my earlier comments in this review).

ping

Disable all flags except -l/-u when bootstrapping