Page MenuHomeFreeBSD

stand: properly declare build dependencies or .WAIT, start building in parallel
ClosedPublic

Authored by kevans on Jan 29 2020, 2:24 AM.

Details

Summary

buildworld already runs the stand build in parallel[1], so make it easier to identify ordering issues by properly establishing dependencies or adding .WAIT where needed.

Everything in stand/ relies on libsa, either directly or indirectly, because libsa build is where the stand headers get installed and it gets linked in most places.

Interpreters depend on their libs, machine dirs usually depend on top-level libs that are getting built and at least one of the interpreter flavors.

For i386, order btx/libi386/libfirewire before everything else using a big-ol-.WAIT hammer. btx is the most common dependency, but the others are used sporadically. This seems to be where the race reporting on the mailing list is- AFAICT, the following sequence is happening:

1.) One of the loaders gets built based on stale btx/btxldr
2.) btx/btxldr gets rebuilt
3.) installworld triggers loader rebuild because btx was rebuilt after

This seems like the most plausible explanation, as they've verified system time and timestamps.

While we're here, let's switch stand/ over to a completely parallel build so we can work out these kinds of issues in isolation rather than in the middle of a larger build.

[1] I'm still working out how it accomplishes this, but reports on the mailing list seem to indicate that some sprinkled SUBDIR_DEPEND fixes the ordering issues; stand/Makefile doesn't seem to be invoked with SUBDIR_PARALLEL defined, but I suspect there's magic afoot arising from the theory that buildworld strictly orders things such that all dependencies should have been built in earlier stages and it may parallelize the rest.

Test Plan

Sweet talk some folks with much better hardware than I can access into running make -C stand -jN w/ maybe a value or two of N >= 32, and make TARGET_ARCH=amd64 buildenv BUILDENV_SHELL="make -C stand -jN"under similar conditions to see if we can reveal any other dependencies/races.

Diff Detail

Repository
R10 FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Adding Simon, too, for some further build insight- it may be more readable to drop a couple of .WAIT hammers in ^/stand/Makefile instead or consolidate the logic somehow, but I'm not sure if .WAIT is a good approach from a build point-of-view when we can probably express the dependencies.

Tested -j1024 /stand builds for powerpc, powerpc64, amd64, i386, aarch64, armv7, and riscv. No failures, significant speed improvement.

This revision is now accepted and ready to land.Jan 29 2020, 3:24 AM

For tree walks .WAIT is a legit solution