release: Avoid pipe in tarball creation

[emaste]

Summary

Previously errors from invoking tar for src.txz or ports.txz were eaten
by the pipeline.  If something fails in creating these tarballs we want
the build to fail rather than producing subtly broken artifacts.  Use
${TAR_XZ_CMD} to add -J to tar's commandline rather than | xz.

[jrtc27]

What shell are we using here, the default system sh? dash doesn't support pipefail, so I'm concerned this will break on Linux.

[emaste]

Yeah it will be default /bin/sh. We could create src.tar and then compress it (although that might be a perf regression depending on how much buffering xz does).

[sjg]

In D50459#1151603, @jrtc27 wrote:

What shell are we using here, the default system sh? dash doesn't support pipefail, so I'm concerned this will break on Linux.

bash is a better option on Linux and supports pipefail, in the past we built ash for Ubuntu - because dash was unusable, but it has gotten harder to build recently, so just using bash is the simple solution.

[emaste]

Is there a reasonable way to have make use bash as the default shell on Linux?

[sjg]

In D50459#1151661, @emaste wrote:

Is there a reasonable way to have make use bash as the default shell on Linux?

Indeed.
.SHELL: name=sh path=/bin/bash

tells it to use bash and assume it behaves like sh. ${.SHELL} will give /bin/bash

You can of course first check whether /bin/sh is bash:

_shell := ${/bin/sh:L:tA:T}
.if ${_shell:Nbash} != ""
# we want bash
.SHELL: name=sh path=/bin/bash
.endif

[emaste]

So perhaps something like:

.if ${.MAKE.OS} == "Linux"
.SHELL: name=sh path=/bin/bash
.endif

need to check if macOS /bin/sh supports -o pipefail or should be included in the condition as well.

@sjg @jrtc27 any thoughts on which approach you prefer (setting .SHELL on Linux vs splitting the tar and xz)?

[jrtc27]

macOS's /bin/sh is governed by /private/var/select/sh. On my system it's /bin/bash and supposedly new macOS installs will default it to /bin/dash (at least according to tools/build/Makefile, which links /bin/bash into the tmp path as sh) so will hit the same issue. Is there a reason to use the pipe though; would it not be even faster, and sidestep the issue entirely, if we just used bsdtar's -J flag? (You can set xz:compression-level and xz:threads on the command line)

[emaste]

would it not be even faster, and sidestep the issue entirely, if we just used bsdtar's -J flag

Good point, although that might be GNU tar on other systems? If so -J should be fine but is --options xz:threads=0 handled?

[jrtc27]

In D50459#1151997, @emaste wrote:

would it not be even faster, and sidestep the issue entirely, if we just used bsdtar's -J flag

Good point, although that might be GNU tar on other systems? If so -J should be fine but is --options xz:threads=0 handled?

TAR_CMD already needs to be bsdtar since we use it with METALOG files in Makefile.inc1 (see fcf1208158973279c6410c1d24233ae8dfaedf91).

[emaste]

TAR_CMD already needs to be bsdtar

Great, thanks. Perhaps I will add TAR_XZ_CMD in share/mk/bsd.own.mk and use that here.

[emaste]

Use tar -J as @jrtc27 suggested

[bz]

LGTM minus the -

release/Makefile

161–162

Do we also want to add the - before cLvf while here and below?