devfs: Keep a Linux controlling tty across the last slave close

[vova_fbsd.ru]

Summary

Linux leaves the session's controlling terminal in place after the last file
descriptor on the slave is closed, so a later open of /dev/tty succeeds.
FreeBSD drops the session vnode reference on that close. /dev/tty then fails,
which breaks Linux programs such as xterm that close the pty slave during startup
and expect /dev/tty to keep working.

When the closer is a Linux process, leave the session reference in place.
Still pass FLASTCLOSE to the driver so the master sees hangup.
Native close behavior is unchanged.

[kib]

Can you provide the self-contained minimal test that I can compile both on FreeBSD and Linux?

sys/fs/devfs/devfs_vnops.c

725–726

If hold_ctty get set to 1, why do we need to ever enter this block?

741

I suggest to handle this differently. Add sv_sysent flag like SV_PERSISTENT_CTTY and test for it there, instead of testing the specific ABI id. Set the flag for all Linux' sysvecs.

[vova_fbsd.ru]

In D60701#1391783, @kib wrote:

Can you provide the self-contained minimal test that I can compile both on FreeBSD and Linux?

ctty-last-close-repro.c2 KBDownload

- usniversal (FreeBSD, Linux) requires libc

ctty-last-close-linux.c3 KBDownload

- linux binary to build and run on freebsd w/o linux_base

% uname -or
FreeBSD 15.1-RELEASE-p3

% ./ctty-last-close-freebsd
before last close: /dev/tty works, session=81221
FAIL: /dev/tty after last slave close: errno=6 (Device not configured)

% ./ctty-last-close-linux
before last close: /dev/tty works
PASS: /dev/tty still opens after last slave close

[vova_fbsd.ru]

devfs: make persistent controlling terminal an ABI property

Add the SV_PERSISTENT_CTTY sysentvec flag for ABIs that retain the
controlling terminal after the last slave descriptor is closed.

Use the flag in devfs_close() instead of checking explicitly for the
Linux ABI, and enable it for all Linux sysentvecs.

This keeps the devfs behavior ABI-agnostic and allows other ABIs to
request the same semantics if needed.

Suggested by: kib
Differential Revision: https://reviews.freebsd.org/D60701

[kib]

sys/fs/devfs/devfs_vnops.c

725–726

So still: hold_ctty can be calculated before entering this block, it is the property of the p/curproc.
And if the hold_ctty is true, we do nothing under the heavy proc_lock+proctree_lock+session lock + devfs_de_interlock+vnode interlock.

[vova_fbsd.ru]

devfs: Avoid ctty teardown locks for persistent ABIs

Determine whether the closing process uses persistent controlling
terminal semantics before entering the ctty teardown path.

Skip the process, proctree, session, devfs, and vnode lock sequence
entirely when SV_PERSISTENT_CTTY is set, since such ABIs do not remove
the session's controlling terminal reference.

The native ctty teardown behavior remains unchanged.

Suggested by:kib
Differential Revision:https://reviews.freebsd.org/D60701