(WIP) Mk/bsd.port.mk: respect sys.mk for {CONFIGURE,MAKE}_ENV

[2khramtsov_gmail.com]

Summary

Note, this is not ready yet; I didn't fully prepare/test reproducers on vanilla ports and OPTIONS,
but I need to paste this because this can help https://wiki.freebsd.org/KubilayKocak/Ports/LinkTimeOptimization
so folks don't need to spend time.

Another note, some ports (e.g. ruby, node) expect that dtrace can find CC etc via "cc" in path.
These need to be patched to respect DTRACEFLAGS (TODO), so I just disabled DTRACE for now.

I fully removed both base LLVM and elftoolchain utils, replacing them with devel/llvm13,
and could build my desktop. My tree (largely WIP and not of upstream quality) is here:
https://codeberg.org/ei/ports/commits/branch/ei I plan to separate commits for vanilla
ports later when I'll have another weekend(s).

---

Mk/bsd.port.mk sources sys.mk (thus make.conf) for toolchain variables:
CC, CPP, CXX, CFLAGS, ... but these are not explicitly passed to ports:

AR, AS, NM, OBJCOPY, OBJDUMP, RANLIB, READELF, SIZE, STRINGS, STRIPBIN.

Mk/bsd.port.mk has {CONFIGURE,MAKE}_ENV for use with targets, which are
ammended with env variables in some (e.g. CROSS_TOOLCHAIN, USE_BINUTILS)
cases.

To respect sys.mk and to avoid breaking these cases, and potentially to
pass the whole toolchain set in one variable, introduce SYSMK_TOOLCHAIN.

It passes environment variable from sys.mk to {CONFIGURE,MAKE}_ENV while
avoiding breakage of CROSS_TOOLCHAIN and USE_BINUTILS. It can be renamed
to e.g. PTOOLCHAIN if CROSS_TOOLCHAIN, USE_BINUTILS, SYSMK_TOOLCHAIN are
unified into one, ammeding only a one set of tools (AR, AS, ... CC, ...)
to {CONFIGURE,MAKE}_ENV, depending on select toolchain by ports or user.

install sources STRIPBIN to choose strip executable when strip is on via
-s flag. To avoid regressing WANT_DEBUG and INSTALL_STRIPPED, source for
STRIPBIN via STRIP_CMD. STRIPBIN is then passed to install via MAKE_ENV.

Direct calls of INSTALL_{LIB,PROGRAM} in ports still need to be prefixed
with ${SETENV} ${MAKE_ENV}, because passing STRIPBIN from bsd.port.mk as
e.g. INSTALL_PROGRAM= ${SETENV} STRIPBIN="${STRIP_CMD}" ${INSTALL} ...
breaks ports that substite install(1) from INSTALL_PROGRAM, e.g. c-ares,
lame, html2text, binutils, gpu-firmware-kmod.

[2khramtsov_gmail.com]

Approximate (WIP) reproducer of my environment (/var/tmp nullfs mounted to pkg repo with devel/llvm13):

poudriere bulk -j 130amd64 -p main -z cc -b latest devel/llvm13

export JAIL_PATH=/usr/local/poudriere/jails/130amd64
export REPO_PATH=/usr/local/poudriere/data/packages/130amd64-main-cc

jail -c path=$JAIL_PATH host.hostname=cc \

mount.devfs \
devfs_ruleset=2 \
mount="$REPO_PATH $JAIL_PATH/var/tmp nullfs ro 0 0" \
command=/bin/sh

cat <<EOF > /etc/pkg/FreeBSD.conf
FreeBSD: { enabled: no }
cc: {

enabled: yes
url: "file:///var/tmp"

}
EOF

pkg install -y llvm13
cd /usr/bin
rm addr2line ar cc c++ cpp gcov ld ld.lld nm objcopy ranlib readelf size strings strip

exit

zfs rename bsd/poudriere/jails/130amd64@clean bsd/poudriere/jails/130amd64@hascc
zfs snapshot bsd/poudriere/jails/130amd64@clean

cat <<EOF > /usr/local/etc/poudriere.d/cc-make.conf
LLVM=/usr/local/llvm13/bin
ADDR2LINE=${LLVM}/llvm-addr2line
AR=${LLVM}/llvm-ar
AS=${LLVM}/llvm-as
CC=${LLVM}/clang
CPP=${LLVM}/clang-cpp
CPPFILT=${LLVM}/llvm-cxxfilt
CXX=${LLVM}/clang++
DTRACEFLAGS+=-x cpppath=${CPP} -x ldpath=${LD}
LD=${LLVM}/ld
LLVM_LINK=${LLVM}/llvm-link
NM=${LLVM}/llvm-nm
OBJC=${LLVM}/clang
OBJCOPY=${LLVM}/llvm-objcopy
OBJDUMP=${LLVM}/llvm-objdump
RANLIB=${LLVM}/llvm-ranlib
READELF=${LLVM}/llvm-readelf
SIZE=${LLVM}/llvm-size
STRINGS=${LLVM}/llvm-strings
STRIPBIN=${LLVM}/llvm-strip
STRIP_CMD=${STRIPBIN}

LLVM_AS_UNSAFE+= firefox
.if ${LLVM_AS_UNSAFE:M${.CURDIR:T}}
AS=
.endif

LLVM_STRIP_UNSAFE+= tcl86
.if ${LLVM_STRIP_UNSAFE:M${.CURDIR:T}}
STRIP= #none
DONTSTRIP= yes
STRIP_CMD= ${TRUE}
.endif

EOF

poudriere pkgclean -A -j 130amd64 -p main -z cc -y
poudriere bulk -j 130amd64 -p main -z cc www/firefox

[2khramtsov_gmail.com]

Combined with D32654, it is possible to have only one (devel/llvm13) LLVM copy on a desktop.

[val_packett.cool]

Another note, some ports (e.g. ruby, node) expect that dtrace can find CC etc via "cc" in path. These need to be patched to respect DTRACEFLAGS (TODO), so I just disabled DTRACE for now.

Hm, I haven’t heard of DTRACEFLAGS, but I have a local patch for using env vars there. And before that I used BINARY_ALIAS for this.

[2khramtsov_gmail.com]

1. Base isn't ready yet (overriding in make.conf causes i.e. include bootstrap issue(s) https://lists.freebsd.org/archives/freebsd-current/2021-December/001116.html) + might need to do something about DTrace first. Base should be ready first before touching ports, the reason I'm abandoning this for now.

2. Passing strip is the hardest: some ports want STRIPBIN, STRIPPROG, STRIP (variable name conflicts with bsd.port.mk). More time needed to get it right (less patching in ports).

3. Need to figure out the right thing to do about ports that substite install(1) from INSTALL_PROGRAM (bsd.port.mk name conflicts with e.g. devel/binutils), maybe rename it.

4. Additional changes to other build systems i.e. Mk/Uses/cmake (-DCMAKE_AR, ...) (and likely others) needed.

5. This has a lot of potential to break ports (i.e. passing AS results in some ports enabling assembly related parts and failing). To have less regressions one may want to first pass AR and do an exp-run, then wait some time for reports, add AS and do another exp-run, etc, then one would need to be a committer to fix breakage in time.

6. This likely needs a redo completely for configure part, etc.

7. I was too high on enthusiasm so posted a premature WIP. I'm too burned out at the moment to work on this, but please, don't consider this effort is blocked on waiting from me (I want to avoid cookie-licking).

On a bright side for koobs@, I did a global LTO build for a small set of ports that I use (~540 total in poudriere). Only 6 failed (e.g. python310) to build with LTO once toolchain is respected.
Before this patch I had ~150 local patches and the fallout from removing base toolchain wasn't over yet.