The goal here is to have a builder with vm.imply_prot_max=1 and kern.elf64.allow_wx=0 set and have opt out for ports which need WX pages for configure/build/stage/tests. This chain isn't exhaustive, after this chain is landed I plan to request exp-run to expose ports that can't build in restricted environment.

I also plan to push elfctl related changes (so after pkg install software should "just work") but I decided that I want to work on proccontrol in ports first (to also be able to bootstrap ports to set elfctl on). JDK ports are an exception and are included in review chain because I've done that before local elfctl changes.

PROCCONTROL_CMD -> PROCCONTROL and put in bsd.commands.mk like elfctl

(also testing git arc)

Harbormaster completed remote builds in B59939: Diff 144722.Oct 12 2024, 4:54 PM

This approach is very intrusive and I don't think the is the right approach, the right one is do mark the binary which are not protmax or wx=0 friendly.
Log ago I have written this: https://reviews.freebsd.org/D43168 for aslr, I think the same approach should be taken for the other security features and it should not be done as wildly as this propose.

In D47086#1074807, @bapt wrote:

This approach is very intrusive and I don't think the is the right approach, the right one is do mark the binary which are not protmax or wx=0 friendly.
Log ago I have written this: https://reviews.freebsd.org/D43168 for aslr, I think the same approach should be taken for the other security features and it should not be done as wildly as this propose.

the right one is do mark the binary which are not protmax or wx=0 friendly

Agreed.

The only way out I can see for ports at the moment is this due to e.g.:

elfctl.mk currently executes post-build not post-stage
- ELF_FEATURES provides opt out for single stage thus ports with special needs (e.g. binaries extracted/generated for tests) may resort to build stages override to issue ${ELFCTL} in port Makefile or patching upstream sources
- ports during stage could generate new binary and overwrite ELF notes which affects build dependencies (e.g. www/node) or bootstraps
bootstraps (e.g. bootstrap-openjdk) may predate NT_FREEBSD_FEATURE_CTL, without proccontrol(1) need re-bootstrapping or hacking ELF binary to have note
ports generating binaries during build stages then executing, see Java again where new javac is built by bootstrap and then launched to build e.g. images during build stage, without proccontrol(1) need patching upstream sources

Patching upstream sources to use elfctl(1) might be futile due:

elfctl(1) opt out being ineffective for some cases with process groups unlike proccontrol(1) where modes are inherited (base design vs. bug?), see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271838
build depending on bootstrap lacking NT_FREEBSD_FEATURE_CTL, thus requiring re-bootstrap and using elfctl(1) on new bootstrap, recurse into the point above with process group opt out
ports may have poor patch upstreaming perspective, see hostile upstreams e.g. www/chromium, and also ports where maintainers are hesitant/not interested to upstream, are unlikely to maintain local patches for strict page permissions which historically have seen low interest from the community and may not justify the rebase churn, leading to strict page permissions having a poor perspective for becoming the default for builders or even base

I fail to see how this could be invasive because:

PROCCONTROL_EXEC would be expanded to empty for ports without USES=proccontrol, thus nothing changes for the majority of ports which don't need WX pages during build
I haven't seen a regression from this or USES=proccontrol

Maybe we could have something similar on poudriere side but then I can't see strict page permissions adoption outside of builders.

2khramtsov_gmail.com added a comment.Oct 16 2024, 12:45 PM

This comment was removed by 2khramtsov_gmail.com.

emaste added a subscriber: jrm.Oct 20 2024, 7:57 PM

In D47086#1074862, @2khramtsov_gmail.com wrote:

elfctl.mk currently executes post-build not post-stage

@brooks describes the reason for this in in D34125. "For general infrastructure, I think you want it to be post-build not post-install because test targets may run on the build tree binary."

This feels like a big hammer.

My most fundamental concern is that this assumes proccontrol flags are heritable and that it's appropriate to disable security functionality at a depth greater than the immediate child. In CheriBSD I am considering adding some flags (which should eventually land in FreeBSD) that exist for developer convenience and aren't inherited. The reason is that there's enough demand for some features that defeat a security mechanism that I think I need to support it, but I want to it be clearly auditable at the binary level which means (due to the shape of the APIs involved) something has to be present in the ELF headers.

I don't think all the concerns about elfctl are particularly well founded. For example bootstrap-openjdk looks easy to fix. I'd support adding an ELF_FEATURES_POST_STAGE or the like to simplify it. Intermediate binaries needing opt-outs would best be discovered and fixed rather than hidden behind a knob at the level of the whole build.

That being said, I don't understand the issues with process groups.

In D47086#1076935, @brooks wrote:

This feels like a big hammer.

Ports needing opt out may use JIT, thus these cases have parts likely written in languages different than C family. These languges may have compilers written in the same language that use JIT, thus multi-stage opt out is likely: for configure (e.g. bootstrap-openjdk), for build (bootstrap-openjdk builds javac which then builds the rest of JDK), for stage (post-built binaries are executed e.g. neovim), and for tests.

Per stage opt out is redundant considering multi-stage opt out is likely until elfctl(1) has broad adoption (with changed behavior to accomodate Bug 271838), changing proccontrol.mk to have per stage opt out would not reduce bsd.port.mk diff and would complicate behavior for nothing, because benefit is still retained when distfile fetch would still be done with strict page permissions as it precedes build stages, and builders can run with strict permissions after port build.

Do {configure,make,install,test} have environment set before *_CMD, and the install stage uses FAKEROOT as parent. Variable bandwagon if a concern could be addressed via e.g. WRAP_{CMD,ENV}. I don't see how PROCCONTROL_EXEC is different.

In D47086#1076935, @brooks wrote:

I don't think all the concerns about elfctl are particularly well founded. For example bootstrap-openjdk looks easy to fix.

See bug 256477 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256477#c20
and bug 271838 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271838#c0

Please propose your solution.

In D47086#1076935, @brooks wrote:

I'd support adding an ELF_FEATURES_POST_STAGE or the like to simplify it. Intermediate binaries needing opt-outs would best be discovered and fixed rather than hidden behind a knob at the level of the whole build.

Build systems (as well as declarative ones e.g. meson) would need to have logic for executing external binaries (e.g. elfctl(1)) post build of a binary with specified name for opt out. Considering the variety in Mk/Uses amount of effort is high, and would still not work port tree wide until ELF opt out behavior in the base is changed to accomodate Bug 271838, meanwhile proccontrol(1) is in all supported FreeBSD versions.

Both elfctl.mk and proccontrol.mk would have no use when build systems adopt elfctl(1) and Bug 271838 is accommodated. If the argument against proccontrol.mk is that opt out should occur intermidiately during build then elfctl.mk is the same as proccontrol.mk as neither can't set ELF notes on intermediate binaries during build stage, build systems can.

Port maintainers with hostile upstreams may also not be interested to maintain local patches, e.g. www/chromium build failures when testing patch rebase cause loss of multi-hour builds due lack of support of incremental builds or ccache in ports:

See bug 244532 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244532#c0
and bug 246245 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246245

In D47086#1076935, @brooks wrote:

In CheriBSD I am considering adding some flags (which should eventually land in FreeBSD) that exist for developer convenience and aren't inherited.

Then consider not using proccontrol for non-heritable flags downstream. I can't be aware of every downstream before I propose a change.

In D47086#1076935, @brooks wrote:

I don't think all the concerns about elfctl are particularly well founded.

"well founded" means "based on facts", see https://dictionary.cambridge.org/dictionary/english/well-founded

I may lose interest if mentioning PRs with elfctl(1) opt out limitations would get comments about whether concerns are based on evidence (PRs are) or not.