Add --symlink-bindir flag to make.py build tool

[jfree]

Summary

Many package managers on macOS now place their bindirs in the /opt
directory. When bootstrapping the FreeBSD build process on macOS,
bmake complains that the /opt/*/bin host tools are not in $PATH. The
included --symlink-bindir flag mitigates this issue by allowing the user
to add directories to $PATH during the host-symlinks target.

[arichardson]

IMO we should add this automatically based on where certain utilities are. Is this for xz? If so make.py could use the result of shutil.which to find that directory.

[jrtc27]

Yeah this isn’t right, PATH just shouldn’t be sanitised when finding binaries on non-FreeBSD, it only makes sense to do on FreeBSD when we know where everything is.

[jfree]

In D37983#862982, @jrtc27 wrote:

Yeah this isn’t right, PATH just shouldn’t be sanitised when finding binaries on non-FreeBSD, it only makes sense to do on FreeBSD when we know where everything is.

Here's a comment from Makefile.inc1:607 that made me publish this patch:

When building on non-FreeBSD we can't rely on the tools in /usr/bin being compatible
with what FreeBSD expects. Therefore we only use tools from STRICTTMPPATH
during the world build stage. We build most tools during the bootstrap-tools
phase but symlink host tools that are known to work instead of building them

Otherwise I would have attempted to unsanitize the $PATH. Perhaps I am misunderstanding
your comment?

[jrtc27]

In D37983#863020, @jfree wrote:

In D37983#862982, @jrtc27 wrote:

Yeah this isn’t right, PATH just shouldn’t be sanitised when finding binaries on non-FreeBSD, it only makes sense to do on FreeBSD when we know where everything is.

Here's a comment from Makefile.inc1:607 that made me publish this patch:

1. When building on non-FreeBSD we can't rely on the tools in /usr/bin being compatible
2. with what FreeBSD expects. Therefore we only use tools from STRICTTMPPATH
3. during the world build stage. We build most tools during the bootstrap-tools
4. phase but symlink host tools that are known to work instead of building them

Otherwise I would have attempted to unsanitize the $PATH. Perhaps I am misunderstanding
your comment?

It should still be sanitised for the build, but not for finding the things to symlink.

[jfree]

In D37983#862949, @arichardson wrote:

IMO we should add this automatically based on where certain utilities are. Is this for xz? If so make.py could use the result of shutil.which to find that directory.

Yes. This was for xz. I feel like adding a shutil.which for every binary that is not in /sbin:/bin:/usr/sbin:/usr/bin is not very robust. If the required host-symlinks
binaries change or if macOS decides to remove some in a future update, we will be responsible for patching make.py. This patch at least allows the user to extend
the $PATH by themselves without modifying scripts.

[jfree]

In D37983#863021, @jrtc27 wrote:

In D37983#863020, @jfree wrote:

In D37983#862982, @jrtc27 wrote:

Yeah this isn’t right, PATH just shouldn’t be sanitised when finding binaries on non-FreeBSD, it only makes sense to do on FreeBSD when we know where everything is.

Here's a comment from Makefile.inc1:607 that made me publish this patch:

1. When building on non-FreeBSD we can't rely on the tools in /usr/bin being compatible
2. with what FreeBSD expects. Therefore we only use tools from STRICTTMPPATH
3. during the world build stage. We build most tools during the bootstrap-tools
4. phase but symlink host tools that are known to work instead of building them

Otherwise I would have attempted to unsanitize the $PATH. Perhaps I am misunderstanding
your comment?

It should still be sanitised for the build, but not for finding the things to symlink.

I don't see how this could be done. $PATH appears to be sanitized before the target is
even entered.

[jrtc27]

The fix is either to not do https://github.com/freebsd/freebsd-src/blob/main/Makefile#L220 on !FreeBSD or to save the original PATH and use it in host-symlinks for finding things. I lean towards the former, though the latter is what we do in CheriBSD today.

[jfree]

In D37983#863026, @jrtc27 wrote:

The fix is either to not do https://github.com/freebsd/freebsd-src/blob/main/Makefile#L220 on !FreeBSD or to save the original PATH and use it in host-symlinks for finding things. I lean towards the former, though the latter is what we do in CheriBSD today.

It looks like a simple

.if ${.MAKE.OS} == "FreeBSD"
PATH=	/sbin:/bin:/usr/sbin:/usr/bin
.endif

fixes the problem. Question is: what side-effects does this have?

[jrtc27]

In D37983#863029, @jfree wrote:

In D37983#863026, @jrtc27 wrote:

The fix is either to not do https://github.com/freebsd/freebsd-src/blob/main/Makefile#L220 on !FreeBSD or to save the original PATH and use it in host-symlinks for finding things. I lean towards the former, though the latter is what we do in CheriBSD today.

It looks like a simple

.if ${.MAKE.OS} == "FreeBSD"
PATH=	/sbin:/bin:/usr/sbin:/usr/bin
.endif

fixes the problem. Question is: what side-effects does this have?

It shouldn't matter much, most of the interesting parts of the build are run with PATH=${TMPPATH}, which is always ${STRICTTMPPATH} for !FreeBSD

[jfree]

In D37983#863030, @jrtc27 wrote:

In D37983#863029, @jfree wrote:

In D37983#863026, @jrtc27 wrote:

The fix is either to not do https://github.com/freebsd/freebsd-src/blob/main/Makefile#L220 on !FreeBSD or to save the original PATH and use it in host-symlinks for finding things. I lean towards the former, though the latter is what we do in CheriBSD today.

It looks like a simple

.if ${.MAKE.OS} == "FreeBSD"
PATH=	/sbin:/bin:/usr/sbin:/usr/bin
.endif

fixes the problem. Question is: what side-effects does this have?

It shouldn't matter much, most of the interesting parts of the build are run with PATH=${TMPPATH}, which is always ${STRICTTMPPATH} for !FreeBSD

Yeah, I just looked it over and you appear to be right. I have created a new review with the proposed changes:
https://reviews.freebsd.org/D37991