Add `pdrfork(RFEMBRYO)`, `pdsetfd` and `pdexec` syscalls

[inquire_JohnEricson.me]

Summary

These syscalls implement a new process creation model that avoids the
fork+exec pattern. I first brought this up a few years ago in
https://lists.freebsd.org/archives/freebsd-arch/2022-January/000140.html
but never got a response. Well, now I am taking matters into my own
hands and just implementing it. :)

(A similar direction is being explored on Linux; compare the proposed
spawn syscall series:
https://lore.kernel.org/all/[e-mail]/; see
where my name is mentioned and "embryonic" is used in particular. The
plan proposed there is to start with that patch series, and then end up
with something like this later.)

pdnew creates an unscheduled process with a loaded executable in a
single operation, returning a process descriptor. pdsetfd installs
file descriptors into the unscheduled process, mirroring pddupfd.
pdstart submits it to the scheduler.

Unlike fork+exec, the new process is never a copy of the parent. It
starts with a fresh fd table, default signal handling, and the
executable already loaded --- there is no intermediate state where the
child runs parent code. This is both simpler and more compatible with
capability-based security (all three syscalls are CAPENABLED).

In order to not have a huge amount of little-tested little-used new
code, effort has been taken to deduplicate this new code with the
traditional fork+exec. But because the division of labor is so
different between the old and new models, this ends up taking quite a
lot of refactoring.

The preceding commit factored the shared logic out of fork1/do_fork
and do_execve into a large number of small helper functions; this
commit generalizes those helpers so each has one old-style caller and
one new-style (pdnew) caller. Because the division of labor is so
different between the old and new models, the new caller drives those
helpers with a *different* process and thread than
curproc/curthread:

• The fork helpers (fork_alloc_proc, fork_register_proc, fork_proc_tree) allocate and register an embryonic process on behalf of a parent that is the caller, not the new process itself.
• The exec helpers activate an image into that embryonic process's address space. Because it is not curproc, string and auxv setup can no longer use plain copyout; the new imgp_copyout, imgp_suword, and imgp_suword32 helpers write into the target vmspace via vmspace_iop() (see the preceding privilege-lifting commit) with a real, process-visible VM_PROT_WRITE.

The embryonic process's p_vmspace starts as NULL. exec_new_vmspace
(called by the image activator) has been modified to handle this,
skipping straight to vmspace_exec() to create the correct vmspace
for the executable's ABI. This avoids allocating a throwaway vmspace.

The embryonic process reuses P_INEXEC (cleared by pdstart) to
protect against ptrace, signal delivery, and other interference while
the process is being set up. Closing the process descriptor before
pdstart destroys the embryonic process via
proc_destroy_embryonic.

Interpreter (shebang) scripts are supported.

The syscalls are documented in pdfork.2 alongside the other process
descriptor syscalls.

Note: This patch series overall is very big, more than I feel competent
reviewing, so I feel a bit ill-mannered submitting it yet. Certainly,
everyone should feel free to ignore it until my "unix socket connectat"
series is landed. I am submitting it now anyway simply because I think
the end functionality is very cool, and I don't want it to grow dusty on
my machine.

Assisted-by: Claude Code (Claude Opus 4.8/5 and Fable 5)

[inquire_JohnEricson.me]

A few fixes

1. Bug fix where we had a corrupted mix of inheriting and non-inheriting

2. Better choice of words for pdfsetfd ("remote" vs "local", not

"parent" vs "child", since the parent is the caller of pdnew). Update
the code and man page both, accordingly.

3. More background info added about inheritting as little state as

possible in the man page.

[kib]

I believe that the userspace API design can be done more ergonomically. Also I believe that there are some issues with VM that we do not want to tackle.

First, I do not think that pdnew() is needed. pdrfork() should be able to do it all, with the addition of some RF* flag(s), similarly how RFSPAWN was added. Basically, a flag should be added that requests a creation of the zigote, which does not yet have the vmspace allocated for it at all. Optionally it might also cause the empty sigacts allocation, and so on.

Then, I strongly suggest not to try to 'exec' an image into the target from the other thread. It is somewhat ingrained into the imgact's and VM operations that we operate on curthread + current vmspace. Instead of trying to audit the whole sys/vm and sys/kern/imgact* to ensure that the assumption is properly removed, add something like pdexec(2). The syscall would unsuspect the zigote process and jumps to a trampoline that 1. creates an empty vmspace (above, pdrfork() created the process without vmspace, or with just empty vm_map) 2. does kern_execve() before first return to userspace.

Perhaps pdexec() should be allowed to either take the path to the executable, or the executable fd like fexecve(2).

I believe this should eliminate most of the 'refactoring', and also makes the proofing the patch for correctness much easier.

[fuz]

It sounds like this could perhaps be simplified by having a pdsyscall system call that executes a system call on the given process descriptor. Then all manipulations on the zygote process just become normal system calls carried out through pdsyscall and activating a process image becomes a call to execve() or similar through pdsyscall. In particular this avoids having to add new stuff every time we come up with something new that we want to set up before sending off the new process.

[inquire_JohnEricson.me]

@kib Interesting. So my first reaction was to be a bit sad, because I personally very much want to "push back on" the fork/exec model, but upon further reflection I think this is not so incompatible after all. Also, you are absolutely right that we need a way to skip the VM refactors for now, as otherwise this stuff is just impossible to review; there just isn't any way around that.

The first thing to note is that even taking my model on its own terms (and wishing away the exec refactor stuff), there is indeed no reason to put the process loading in the creation call itself. Indeed, it is arguably a departure from the "builder pattern" to do so. So that part is totally fine. That would leave us with pdnew, pdinitvm, and pdstart, where the setting of the FDs can come either before or after pdinitvm.

The second part is that if we do have a "regular" pdexec to start with, we can imagine giving it a PD_NOSTART flag later (if we want) and then pdinitvm() = pdexec(PD_NOSTART). That's nice --- my making mincemeat of exec is not something that can *never* happen; it is just kicked (arbitrarily far) down the road.

Similarly, we can have pdnew() = pdrfork(RFEMBRYO), which is basically just what you said. As long as we have a way for the embryo to inherit as little state from the parent as possible (which I think is an important capability-mode design principle --- sharing should be opt-in, not opt-out), that is fine too.

The only question remaining for me is whether RFEMBRYO should be handled as another flag in fork1, or whether we should continue to factor out the little functions and have a separate kernel pdnew function that the syscall wrapper would delegate to, for a "composition over configuration" approach. I will try both approaches, since I think the fork1 mincemeat isn't nearly as bad as the exec mincemeat.

---

@fuz You may be interested in https://github.com/catern/rsyscall, which I referenced in my original 2022 freebsd-arch email linked in the patch description. It adopts the same philosophy.

[kib]

In D58688#1346717, @inquire_JohnEricson.me wrote:

@fuz You may be interested in https://github.com/catern/rsyscall, which I referenced in my original 2022 freebsd-arch email linked in the patch description. It adopts the same philosophy.

We have ptrace({PT_SC_REMOTE). Allowing remote syscalls as a generic facility without attaching to the target as debugger is IMO not acceptable.

[fuz]

In D58688#1346727, @kib wrote:

In D58688#1346717, @inquire_JohnEricson.me wrote:

@fuz You may be interested in https://github.com/catern/rsyscall, which I referenced in my original 2022 freebsd-arch email linked in the patch description. It adopts the same philosophy.

We have ptrace({PT_SC_REMOTE). Allowing remote syscalls as a generic facility without attaching to the target as debugger is IMO not acceptable.

Sure, we can instead build 200 something pd variants of existing syscalls for every single thing you might want to set up in a nascent process. Or go through the klunky ptrace interface. Or perhaps implement pdsyscall() as a convenience wrapper around ptrace(PT_SC_REMOTE) to make it less awkward to use. Whatever makes this interface not dead on arrival (i.e. unusable as soon as you want to do something slightly nontrivial).

[fuz]

In D58688#1346727, @kib wrote:

In D58688#1346717, @inquire_JohnEricson.me wrote:

@fuz You may be interested in https://github.com/catern/rsyscall, which I referenced in my original 2022 freebsd-arch email linked in the patch description. It adopts the same philosophy.

We have ptrace({PT_SC_REMOTE). Allowing remote syscalls as a generic facility without attaching to the target as debugger is IMO not acceptable.

On second though, ptrace itself is not a good solution as ptrace can be turned off for the whole system through security.bsd.allow_ptrace=0, which would then consequentially break the whole interface idea.
Jan Bramkamp proposes that this could be addressed by allowing a potential pdsyscall to only work while the target process is in embryonic state or permission is granted some other way (e.g. we currently debug it or we have that capability). This would avoid the security problems as running a system call in an embryonic process is analogous to running it in a fork, which is already possible.

[kib]

In D58688#1346730, @fuz wrote:

In D58688#1346727, @kib wrote:

In D58688#1346717, @inquire_JohnEricson.me wrote:

@fuz You may be interested in https://github.com/catern/rsyscall, which I referenced in my original 2022 freebsd-arch email linked in the patch description. It adopts the same philosophy.

We have ptrace({PT_SC_REMOTE). Allowing remote syscalls as a generic facility without attaching to the target as debugger is IMO not acceptable.

On second though, ptrace itself is not a good solution as ptrace can be turned off for the whole system through security.bsd.allow_ptrace=0, which would then consequentially break the whole interface idea.
Jan Bramkamp proposes that this could be addressed by allowing a potential pdsyscall to only work while the target process is in embryonic state or permission is granted some other way (e.g. we currently debug it or we have that capability). This would avoid the security problems as running a system call in an embryonic process is analogous to running it in a fork, which is already possible.

This might be, but there is a lot of signicant problems with either approach to completely open pdsyscall:

• it indeed cannot be reliably implemented for already running process, since we cannot safely execute syscall handlers in arbirary contexts inside kernel. And it is probably quite hard to formulate when we can safely do that without issues. Like, we might postpone the execution until the process reaches a kernel/user boundary like AST, but there is no bound time where it happen at all. Also, I suspect it is not safe in the sense that it would unknowingly interact with the syscall results we are returning when AST is processing.
• if we allow pdsyscall() only for zigote/embrionic state, then the problem is that the pdnew() or pdrfork(RFZIGOTE) or whatever must create a process without vmspace constructed and a lot of kernel state not yet initialized. The whole goal of this exercise is to avoid some spawn overhead inherently present in the fork/exec model. But then, if the kernel state is not present, we cannot safely allow a syscall handler to run.

So I am skeptical to the pdsyscall() idea. PT_SC_REMOTE does not exhibit the problems because ptracestop() is designed to provide the known stable process state for debuggers.

[kib]

Some discussions in the nearby camp https://lwn.net/Articles/1076018/

[inquire_JohnEricson.me]

Update to be much closer to what @kib requested.

Granted, I still have the forking refactors, because to me fork(RFEMBRYO) isn't really a "fork", since the goal is to avoid inheriting as much parent process state as possible. It is fine to reuse the syscall, but I don't want the choice of reusing the syscall to leak into the internals in contrast to the above reasoning. We can revisit this though.

However, the much-larger exec refactors are now gone, and this just execs the "normal way", as initiated by the trampoline. The VM logic is untouched, and the abandoned revisions doing those refactors I will (after I submit this new diff) no longer mark as ancestors of this revision.

On top of this I do in fact now have a posix_spawn implementation --- see https://github.com/obsidiansystems/freebsd-src/commits/test-utils-better-proc-spawn/, but I am still reviewing and testing it, so I am not submitting new phab revisions for the subsequent commits at this time.

[inquire_JohnEricson.me]

Also, there is the very pressing question of whether "embryo" or "zygote" is more apt :).

I am persuadable that a process is a "zygote" before it has a VM space, and an "embryo" after. That means that with @kib's design, we would call pre-exec'd process a "zygote" instead, as appears to be his preference.