This differential adds Solaris doors IPC support to the kernel for amd64 architectures. Solaris doors are a form
of lightweight inter-process communication which allows a thread to invoke a function
from another process, essentially mimicking a local remote procedure call.
A serving process may expose its service by creating a door that should then be attached to a file.
A client may then invoke the exposed procedure using a "door call" operation on the attached file, during
which the kernel chooses a thread in the serving process which will execute the procedure. The serving
thread will invoke a "door return" operation upon returning which will transfer any results to the calling thread.
The kernel maintains a list of doors for each process and a per-process door server thread pool, with the
possibility of each door having its own private thread pool. Currently, all door-related operations are done
via the "door" syscall which wraps several syscalls using subcodes.
When a door is created, the basic library spawns a new thread that invokes the "door_return" syscall.
This thread is then placed on a door thread pool and waits indefinitely.
When a "door call" syscall is invoked, the kernel picks a server thread from a thread pool and transfers any
arguments provided by the calling thread to the serving thread. Small amounts of data are simply copied,
while larger arguments are transferred using shared mappings of argument data. The calling thread then waits for results.
The selected server thread gets woken up and its frame is adjusted so the procedure attached to the invoked door gets executed
when we return to userspace from "door_return". When the procedure is finished, the thread invokes "door_return" once again, and
any results are transferred to the calling thread, which is then unblocked and returns to userspace.
Currently, doors are attached to an existing file by "latching" onto its vnode, replacing its vnops vector with
the "doorfs_vnops" vector. A new vnode, file type and vnops vector was added to achieve this
(VDOOR, DTYPE_DOOR, and doorfs_vnops respectively). The owning process may attach or detach the
door to or from a regular file using "fattach" and "fdetach", which are implemented as door syscalls.
Currently, the only unresolved issue is that there is no handling of thread termination due to signals
which can lead to kernel memory leaks.