unix: allow `connectat(2)` to name the peer socket by descriptor

[inquire_JohnEricson.me]

Summary

Accept an empty sun_path when fd is not AT_FDCWD: the descriptor
then names the peer unix socket directly, instead of being the starting
directory for a pathname lookup. The held file reference keeps the peer
PCB stable, playing the role unp_vp_mtxpool plays in the pathname
path.

The descriptor must carry CAP_CONNECTAT and refer to an AF_UNIX
socket (EPROTOTYPE otherwise, ENOTSOCK for non-sockets). As with a
pathname, a stream/seqpacket peer must be listening. No filesystem
permission or MAC vnode check applies on this path: possession of the
descriptor is the authorization, as with descriptor passing.

Note this makes it possible to connect a datagram socket to an unbound
peer, which no pathname could previously name.

connect(2) and the implicit-connect send path pass AT_FDCWD and
still reject an empty path with EINVAL.

The unp_sun_path() call is hoisted out of unp_connectat() because the
early exit conditions for the two system calls (connect(2) and
connectat(2)) are slightly different.

Additionally, support /dev/fd/<N>. In a world with connectat(2),
this is largely overkill, but this also allows me to add support for
direct peer connections with plain connect(2). I think that is a wise
choice because this will allow me to propose this functionality for
Linux too without a new system call (saving that conversation for
later). Ultimately, I want to see multiple operating systems support
this to foster broader userland adoption, which should benefit everyone
including FreeBSD --- it's nicer if more 3rd party in addition to 1st
party software uses the new kernel functionality. Therefore, I hope this
additional feature is also acceptable.

That said, I am a little confused by fdescfs's myriad modes, and when it
is the responsibility of the system call vs namei to handle O_PATH /
vnode descriptors. So I think I will need some advice on this part of
the implementation.

Signed-off-by: John Ericson <[e-mail]>
Assisted-by: Claude Code (Claude Opus 4.8 and Fable 5)

[markj]

sys/kern/uipc_usrreq.c

3013

I am not sure if it is safe to drop the ref while potentially holding unp2 locked. What if it turns out to be the last ref on so2, and we end up calling uipc_close() on it? We could recurse on the lock.

Hmm I guess that can't happen since we can only arrive here with return_locked == false. In that case it'd be best to assert that. Or go even further and find a way to refactor this such that the flag isn't needed.

3014

Let's please rename the label to avoid suggesting it's an error path. out is fine.

[markj]

This would also need updates to connectat.2 and unix.4. I also wonder how this interacts with the abstract unix socket namespace in Linux? I don't remember the details of how that works, I guess this is complementary and doesn't make that harder to implement?

Finally, when exactly is this useful?

[inquire_JohnEricson.me]

In D58405#1340280, @markj wrote:

This would also need updates to connectat.2 and unix.4.

Oh yes, can do. Should that go right in this diff?

when exactly is this useful?

I would follow it up with another patch to allow listen *without* first bind. Then, we can have the entire listening socket workfow without touching the file system at all.

I want to do sort of "super socket activation" with this, where processes are given their own listening sockets (like normal socket activation) and their peers, but their peer ones have only have CAP_CONNECTAT.

The idea is then you have a completely capability-based system of communicating processes, where processes are only started with the peers they need to know about / peer socket existing is disclosed on a need-to-know basis.

I also wonder how this interacts with the abstract unix socket namespace in Linux? I don't remember the details of how that works, I guess this is complementary and doesn't make that harder to implement?

Hehe I have also been looking at implementing it on Linux. Here are some threads building up to that:

• https://lore.kernel.org/all/[e-mail]/
• https://lore.kernel.org/all/[e-mail]/:
• https://lore.kernel.org/all/[e-mail]/

For background, I am very much not a fan of that abstract namespace, as it appears to me to be simply inventing one form of global namespacing to replace another (the VFS). I like this much better, and wish the unices had gone this route all along when *at and the "file descriptor revolution" hit the scene.

That said, the existence of abstract sockets pose no problem in implementing this on Linux. The only problem the goto restart I talk about in the third linked thread. And even that, I can work around, just with more code churn.

[kib]

sys/kern/uipc_usrreq.c

3111

Is EMPTYPATH case is ever possible after the change?

If not (and this is my understanding), it was possible to path O_PATH-opened fd specifying the unix domain socket, and now it is not.

[inquire_JohnEricson.me]

sys/kern/uipc_usrreq.c

2940–2941

@kib My understanding is that because of this line, the EMPTYPATH was unfortunately in vane. I agree it should work: I would be happy to add some tests for this case and fix it.

[markj]

In D58405#1340290, @inquire_JohnEricson.me wrote:

In D58405#1340280, @markj wrote:

This would also need updates to connectat.2 and unix.4.

Oh yes, can do. Should that go right in this diff?

Yes, that's fine.

when exactly is this useful?

I would follow it up with another patch to allow listen *without* first bind. Then, we can have the entire listening socket workfow without touching the file system at all.

I want to do sort of "super socket activation" with this, where processes are given their own listening sockets (like normal socket activation) and their peers, but their peer ones have only have CAP_CONNECTAT.

The idea is then you have a completely capability-based system of communicating processes, where processes are only started with the peers they need to know about / peer socket existing is disclosed on a need-to-know basis.

Why can't the connections be pre-established using socketpair(2)?

I also wonder how this interacts with the abstract unix socket namespace in Linux? I don't remember the details of how that works, I guess this is complementary and doesn't make that harder to implement?

Hehe I have also been looking at implementing it on Linux. Here are some threads building up to that:

• https://lore.kernel.org/all/[e-mail]/
• https://lore.kernel.org/all/[e-mail]/:
• https://lore.kernel.org/all/[e-mail]/

For background, I am very much not a fan of that abstract namespace, as it appears to me to be simply inventing one form of global namespacing to replace another (the VFS). I like this much better, and wish the unices had gone this route all along when *at and the "file descriptor revolution" hit the scene.

That said, the existence of abstract sockets pose no problem in implementing this on Linux. The only problem the goto restart I talk about in the third linked thread. And even that, I can work around, just with more code churn.

Ok. I was really asking because I don't want to preclude porting abstract sockets to FreeBSD. :)

[kib]

sys/kern/uipc_usrreq.c

2940–2941

Meh.

But then, why not just not jump to 'bad' if getsock() failed, instead of falling to the namei() path? I suspect it is enough.

[inquire_JohnEricson.me]

Major overhauls, now that this is part of a much larger patch series

Tried to fix all outstanding feeedback in the patch series, along with
also implementing /dev/fd/<N> support for the reasons given in the new
summary.

[inquire_JohnEricson.me]

Expanded test suite to cover the new larger feature surface area

[inquire_JohnEricson.me]

Oh I did the wrong diff for this last update?

[inquire_JohnEricson.me]

sys/kern/uipc_usrreq.c

3013

D58460, which is earlier in the now-longer patch stack, seeks to addresses this more in isolation

[markj]

In D58405#1341218, @inquire_JohnEricson.me wrote:

Major overhauls, now that this is part of a much larger patch series

Sorry, I still do not quite get the point of this. Why can't the server which hands out a listening socket to other processes to connect to, instead hand out pre-connected sockets?

[inquire_JohnEricson.me]

Sorry, I still do not quite get the point of this. Why can't the server which hands out a listening socket to other processes to connect to, instead hand out pre-connected sockets?

Sorry, I'm a bit confused by this question. I indeed want to hand out listening sockets for other processes to connect to! I just want to do so with file descriptors rather than global namespaces (filesystem or Linux abstract socket namespace).

[markj]

In D58405#1341700, @inquire_JohnEricson.me wrote:

I indeed want to hand out listening sockets for other processes to connect to! I just want to do so with file descriptors rather than global namespaces (filesystem or Linux abstract socket namespace).

Right, my point is that e.g., socketpair() gives you a pair of preconnected sockets, no namespace access needed. Rather than handing over a listening socket, one can instead hand over one end of a pre-connected socket pair. This is a bit less flexible than handing over a listening socket, but I can't see how the proposed feature enables anything that can't be achieved with existing interfaces. What am I missing?

[inquire_JohnEricson.me]

Ah OK, I understand now.

Yes, that's true. In fact, if we have a socket pair that we only use for SCM_RIGHTS, and "clients" only create socket pairs and send one half over the original socket pair, then we've exactly reinvented listening sockets!

So why not do that? Well, with this project of mine I've already committed to patching all the client programs. I rather *not* also have to patch all the server programs. Lots of software supports socket-activation already, and I would like to use that software unmodified — this is possible because that software doesn't care whether or where the sockets are bound or not, as long as they can call listen.

Also, many such servers benefit from "TCP/Unix polymorphism": they want to be able to listen and accept without caring what the underlying protocol is, so long that it is connection oriented. (I've elsewhere argued that this also is a good reason for QUIC in kernel space, orc something like FUSE for the socket API.) listening sockets might be superfluous for unix domain sockets, but they aren't for TCP, so even if we were inventing everything from scratch, I would think we would still want to support listen and accept for unbound Unix sockets. Protocol polymorphism is just too good to give up.

I want to try to upstream patches supporting capability mode (with my new system calls) where possible. (This is partly why I am trying to get this change and my process spawning one in Linux and, eventually, WASI, not just FreeBSD.) You could say that I'm taking another crack at @ed's CloudABI of you squint. So having enough "niceness" on having a chance at selling 3rd party software on my patches is very important to me.

---

I'm one of the maintainers of the the Nix package manager, and our daemon is such a protocol-polymorphic socket-activated server. While this is one case where I could probably upstream the patch either way, I know my comaintainers will be happy if we just don't need to :).

[inquire_JohnEricson.me]

@markj. are you still on the fence for the motivation after what I wrote? If so, might it make sense to first look at some of the non-functional changes before this one? I am hoping that they would stand on their own, especially D58460.

[markj]

In D58405#1342710, @inquire_JohnEricson.me wrote:

@markj. are you still on the fence for the motivation after what I wrote? If so, might it make sense to first look at some of the non-functional changes before this one? I am hoping that they would stand on their own, especially D58460.

I think the proposal as a whole is reasonable. This new behaviour for connectat() feels very similar to AT_EMPTY_PATH, so there's some precedent for it. It'd help me to see some more concrete examples of how it's useful, and it's not not clear to me yet whether Linux is planning to adopt connectat() or whether it's going with a different interface.

I'll try to go over the other reviews this week, it seems worthwhile to try and give this code more structure, indeed.

[inquire_JohnEricson.me]

I think the proposal as a whole is reasonable. This new behaviour for connectat() feels very similar to AT_EMPTY_PATH, so there's some precedent for it.

Yay glad to hear! That is indeed the precedent as I see it too.

It'd help me to see some more concrete examples of how it's useful

Right now the code is all in my head, but I can whip something up. For now, if it helps, my plan includes proposing something for systemd if I can get it in linux.

and it's not not clear to me yet whether Linux is planning to adopt connectat() or whether it's going with a different interface.

My guess is int unix_connectat(int fd, const char *path, int flags), possibly two sets of flags with one matching openat2. My next step there is a kernel-only interface so there is less pressure to get everything right the first time with the stable UABI. I did in fact find a nice use-case in the kernel for this by accident.

I'll try to go over the other reviews this week, it seems worthwhile to try and give this code more structure, indeed.

Also glad to hear this! I'm glad we feel the same way regarding structure and small functions, not everyone does!