Rework the world subdir build targets to use the standard SUBDIR_PARALLEL mechanism.
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778. Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization. Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.
I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/ before the rest of the system (described in r289433), and etc/ being last (described in r289435), is all that matters. So now a .WAIT is added in the proper places when invoking any 'install*' target. A parallel installworld does work and took 46% of the time a non-parallel install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel. A 'par-all' target is still used by the 'everything' stage of buildworld to continue building in parallel as it already has been. This works because most of the dependencies are handled by the early bootstrap phases as well as 'libraries' and 'includes' phases. This lets all of the SUBDIR build in parallel fine, such as bin/ and lib/. This will not work if the user invokes 'all' though as we have dependencies spread all over the system with no way to depend between them (except for the dirdeps feature in the META_MODE build). Calling 'make all' from the top-level is still useful at least when using SUBDIR_OVERRIDE.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division