Add a BUILD_WITH_OPIPEFAIL option to use `bash -o pipefail` as the bmake shell

[arichardson]

Summary

This is useful to find failed commands if the output is piped. By default
shells will only return the exit code of the last command in the pipeline.
The -o pipefail option causes the pipelline exit code to be nonzero if any
of the intermediate commands return non-zero.
This has been extremely useful for me when debugging build failures on
non-FreeBSD hosts due to missing/incompatible commands.

This currently creates a sh binary that will execute bash -o pipefail but
once we have -o pipefail support in /bin/sh we could instead bootstrap a
version of /bin/sh that defaults to pipefail.

[brooks]

FreeBSD's sh grew pipefail support in rS344502 so this might be OBE.

[bdrewery]

Has this turned up any bad pipelines in Makefiles?

[bdrewery]

In D16868#415150, @bdrewery wrote:

Has this turned up any bad pipelines in Makefiles?

Ah right there is that dd one from a while back.

Now that pipefail is in we could do this a lot simpler like tools/install.sh works.
I don't think it needs to be a MK_ option.
It doesn't need to install.
Maybe something like this could work in tools/sh.sh that we just set with __MAKE_SHELL=?

#! /bin/sh
set -o pipefail 2>/dev/null || :
exec "$@"

[bdrewery]

In D16868#415167, @bdrewery wrote:

In D16868#415150, @bdrewery wrote:

Has this turned up any bad pipelines in Makefiles?

Ah right there is that dd one from a while back.

Now that pipefail is in we could do this a lot simpler like tools/install.sh works.
I don't think it needs to be a MK_ option.
It doesn't need to install.
Maybe something like this could work in tools/sh.sh that we just set with __MAKE_SHELL=?

#! /bin/sh
set -o pipefail 2>/dev/null || :
exec "$@"

Hmm set -o pipefail || : doesn't actually ignore the error.

# sh -ec 'set -o pipefail || :; echo test'
set: Illegal option -o pipefail

:(

[bdrewery]

In D16868#415222, @bdrewery wrote:

In D16868#415167, @bdrewery wrote:

In D16868#415150, @bdrewery wrote:

Has this turned up any bad pipelines in Makefiles?

Ah right there is that dd one from a while back.

Now that pipefail is in we could do this a lot simpler like tools/install.sh works.
I don't think it needs to be a MK_ option.
It doesn't need to install.
Maybe something like this could work in tools/sh.sh that we just set with __MAKE_SHELL=?

#! /bin/sh
set -o pipefail 2>/dev/null || :
exec "$@"

Hmm set -o pipefail || : doesn't actually ignore the error.

# sh -ec 'set -o pipefail || :; echo test'
set: Illegal option -o pipefail

:(

This works and avoids a fork and error.

x=$(set -o); case $x in *pipefail*) set -o pipefail ;; esac

I'm trying to avoid adding fork/exec/grep overhead to a sh_pipefail.sh script since it will be executed a lot.

[arichardson]

In D16868#415223, @bdrewery wrote:

In D16868#415222, @bdrewery wrote:

In D16868#415167, @bdrewery wrote:

In D16868#415150, @bdrewery wrote:

Has this turned up any bad pipelines in Makefiles?

Ah right there is that dd one from a while back.

Now that pipefail is in we could do this a lot simpler like tools/install.sh works.
I don't think it needs to be a MK_ option.
It doesn't need to install.
Maybe something like this could work in tools/sh.sh that we just set with __MAKE_SHELL=?

#! /bin/sh
set -o pipefail 2>/dev/null || :
exec "$@"

Hmm set -o pipefail || : doesn't actually ignore the error.

# sh -ec 'set -o pipefail || :; echo test'
set: Illegal option -o pipefail

:(

This works and avoids a fork and error.

x=$(set -o); case $x in *pipefail*) set -o pipefail ;; esac

I'm trying to avoid adding fork/exec/grep overhead to a sh_pipefail.sh script since it will be executed a lot.

I wonder if we should just add an environment variable such as "SH_ENABLE_PIPEFAIL" and use that in a bootstrapped bin/sh to enable pipefail? That would avoid the fork() overhead. And as a fallback we could invoke the wrapper script/binary.

[brooks]

Bring in Jilles to comment on the idea of adding an environmental variable to enable pipefail.

[bdrewery]

Jilles mentioned on IRC a simpler way in the wrapper to enable it without the set -o call:

command set -o pipefail 2>/dev/null || :