There is no need in the underscore prefix. The symbol is static.

Why you cannot use sigset_t ?

SIGCANCEL should be added unconditionally to the set.

SIG_DFL can be ignored as well. You may also clear bits when SIG_IGN and SIG_DFL are specified.

Clearing bits may be incorrect when two threads issue sigaction for the same signal at the same time, one setting a handler and one setting the default action. libthr does not guarantee that the previously installed handler cannot be called after a sigaction with SIG_DFL returns, but does appear to lock the change properly otherwise.

I hadn't spent any time considering any concurrency issues that may happen here, so I opted for the bare minimum uint32[_SIG_WORDS] that I can operate on atomically -- a luxury not provided by sigset operations last I had looked.

I thought I had a memset here -- must have been lost in transit across machines. =( Will address this in another revision.

The initial concern I had is invalid when I address your concerns about using libthr versions of sig* operations, but I did not want to clear these initially because I didn't want to clobber _used_sigs in the vfork parent when the child resets everything.

Is there any reason I shouldn't have __libc_used_signal_reset just reset these ones unconditionally?

It might be wise to fall back to vfork() on EINVAL.

You should check what you stated. If RFSPAWN is specified, all other flags must be cleared.

if (fr->fr_flags2 & FR2_SPAWN) == 0)

That said, I do not think it is correct. You must copy and then reset sigactions to have any sane semantics. Most important, the already queued signals should be dropped, if any.

Better call the flag basing on its functionality, not use.

/* kernel: parent sleeps ...

/* user: vfork(2) ...

Right, also going to revert the KERNELFLAG change and special case it here FOR RFPSPAWN

Perhaps explain that this happens on old kernels.

I prefer to put {} around branches consistently: if one branch requires it, then the other also get {}.

Your sig_drop_caught() requires p2->ps_mtx locked around the call.

Add asserts that both proc lock and ps_mtx are locked.

I'm surprised that this works at all on (among others) amd64: the child pops the return address and overwrites it with new return addresses for calls it makes; how can the parent's return (the ret instruction in rfork.S from SYS.h) go to the right place? Note that vfork has a hand-written stub that deals with this problem.

On architectures such as sparc64, arm and mips where subroutine return addresses are stored in a register and system call stubs do not use the stack, this should not be an issue.

Indeed, perhaps rfork_thread(3) should be used. I hope that it still works.

I'll double check my test system and rework this

Using rfork_thread(3) to implement something vfork-like does not seem a violation of its deprecation. The deprecation is about using rfork_thread(3) to create threads.

rfork_thread(3)'s API is deficient in specifying the stack: it only takes an initial stack pointer, where I would expect a base and a size. Perhaps because of that or because it is deprecated, it is only available on i386 and amd64.

Using a separate stack for the child process does avoid GCC bugs like https://gcc.gnu.org/bugzilla/show_bug.cgi?id=21161.

By 'hope' I mean that we do not corrupt userspace registers in the syscall for rfork_thread() to work. Note that fast syscall on amd64 does not restore all registers as an optimization.

Fortunately, child return from fork sets PCB_FULL_IRET in cpu_fork(), so all registers are restored. From my reading of the rfork_thread() code, it does not matter for parent.

But I did not verified it by a test.

rfork_thread(RFPROC | RFFDG | RFMEM, ...) appears to work still (on stable/11 amd64 and head amd64).

rfork_thread.S only depends on %rsi being preserved in the child, but vfork.S depends on %rsi being preserved in both parent and child. The fast return path of fast_syscall additionally preserves %rdi and %rsi, calling that a bonus in comments. I don't think it is a bonus -- vfork.S depends on it and has no good alternative location to store the parent's return address (thread local storage? yuck).

From my reading, the child dependencies are %rsi, %r12, and %rbx. But they are handler by iret path.

You do not need both defines.

On amd64, there is so called red zone, an area in the stack below %rsp that compiler is free to use. Red zone size is 128 bytes, which leaves 192 - 128 = 64 bytes to use until formal overflow. IMO it is too optimistic.

Ah, that's good to know- 192 was a rough approximation based on measured-once...

do_posix_spawn -> process_spawnattr -> (syscalls)/sigismember

do_posix_spawn -> process_file_actions -> process_file_actions_entry -> (syscalls)

So accounting for 3/4-deep, perhaps 768 as a conservative estimate? Even 1k wouldn't exactly hurt anything, that's 'small potatoes' as far as posix_spawn memory requirements go, I would think.

Various exported functions such as _execve() are called, which might cause rtld and/or libthr to be involved and stack usage to go up greatly. In principle, it's possible to do the necessary work with only unexported aliases and/or do preparations like getenv("PATH") in the parent process.

Do you mean "must not return"?

Use 1U << 31 while there.

Hmm... at least most of this should be successfully avoiding libthr, as it's "in an undefined state after the vfork" according to it.

What specifically would you propose here? Switching _execve over to the unwrapped version and using an even more conservative 1k/2k/4k for the stack size to catch any edge cases?

Hmmm... I caught that once, but it looks like I had only fixed it locally in the diff that I uploaded (61840)... fixed for next update.

Alternatively, how bad of an idea would it be to fetch the stack pointer, subtract past the red zone, and use that for the rfork_thread stack? This is only x86 that has this problem, so it's really tempting.

I would go forward with 2K or even 4K stack and claim that the issue is resolved.