ttydev_open: spray tty_gone checks

[avg]

Summary

Even we may not care whether some ttydevsw_* calls succeed or not,
every of such calls may drop t_mtx and the tty state may change.
That is particularly true for USB-based drivers such as umodem.
Then, every ttydevsw_* dispatch function asserts that the terminal is
not gone.

Those things combined may lead to triggering the assert with
INVARIANTS-enabled kernel.
Example:

panic: Assertion !tty_gone(tp) failed at /usr/src/sys/sys/ttydevsw.h:165
cpuid = 3
time = 1780348587
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff80647b3b =
db_trace_self_wrapper+0x2b/frame 0xfffffe059c07f730
kdb_backtrace() at 0xffffffff809d12e7 = kdb_backtrace+0x37/frame 0xfffffe059c07f7e0
vpanic() at 0xffffffff809803b9 = vpanic+0x169/frame 0xfffffe059c07f920
panic() at 0xffffffff80980193 = panic+0x43/frame 0xfffffe059c07f980
ttydevsw_modem() at 0xffffffff80a0f2e7 = ttydevsw_modem+0x37/frame 0xfffffe059c07f990
ttydev_open() at 0xffffffff80a0e7ba = ttydev_open+0x2ba/frame 0xfffffe059c07f9e0
devfs_open() at 0xffffffff80818e49 = devfs_open+0x129/frame 0xfffffe059c07fa40
VOP_OPEN_APV() at 0xffffffff80dbb403 = VOP_OPEN_APV+0x93/frame 0xfffffe059c07fa70
vn_open_vnode() at 0xffffffff80a890b1 = vn_open_vnode+0x1e1/frame 0xfffffe059c07fb20
vn_open_cred() at 0xffffffff80a88a6c = vn_open_cred+0x58c/frame 0xfffffe059c07fc90
openatfp() at 0xffffffff80a7ef47 = openatfp+0x257/frame 0xfffffe059c07fdd0
sys_openat() at 0xffffffff80a7eccd = sys_openat+0x3d/frame 0xfffffe059c07fe00
amd64_syscall() at 0xffffffff80d2816b = amd64_syscall+0x18b/frame 0xfffffe059c07ff30
fast_syscall_common() at 0xffffffff80cfd54b = fast_syscall_common+0xf8/frame 0xfffffe059c07ff30
--- syscall (499, FreeBSD ELF64, openat), rip = 0x82523a41a, rsp =
0x8209237b8, rbp = 0x820923830 ---

Additionally, ttydev_open could return success for an already gone TTY.
Not sure if that was a problem.

I am not fond of this approach to fixing the issue.
In my opinion, it may be better to change ttydevsw_* to check tty_gone
and return ENXIO instead of asserting.

With the asynchronous nature of USB and a peripheral disconnect not
being unlikely, it's hard or even pointless to keep the assert working.

[avg]

To expand a little on the commit message, I think that commit 36a80f4264350 laid ground for always detecting the situation where a TTY is gone because of USB serial dropping t_mtx.
But it turns out that not all ttydevsw_* calls are error-checked.
I think that in some cases we may legitimately want to ignore some errors, but ENXIO is not such an error.
There were several possible ways to fix the issue, I picked one and described another alternative, but I am open to other suggestions.

[kib]

sys/kern/tty.c

353

While waiting, could CLOCAL flag change under us?

[kevans]

I am not fond of this approach to fixing the issue.
In my opinion, it may be better to change ttydevsw_* to check tty_gone
and return ENXIO instead of asserting.

FWIW, I'd prefer handling this in ttydevsw_* as well, and probably amending the comment at the top of sys/ttydevsw.h or just above ttydevsw_open() to note that callers should avoid assuming that hardware didn't drop the lock at all.

I think tty_gone is the only case *we* need to worry about- termios doesn't say anything about tcsetattr and friends in multithreaded program, but it's hard to reason about how they'd behave so I think you have to assume userland would construct its own mutex for that.

sys/kern/tty.c

353

Shouldn't be able to because the tty isn't opened yet for a concurrent thread to alter it, and the various drivers shouldn't be changing termios flags here

[avg]

Okay, let me re-work the patch.
Not sure if it's better to do here or just open a new review since the code would be completely different.

[avg]

FWIW, I also have an unrelated question / concern.

It seems that ttydev_open activates DTR+RTS signals even before calling ttydevsw_open`.
In the embedded world, those signals are often translated to a micro-controller reset.
Or firmware is waiting for those signals to start running interesting things.
In my experience, the very early output from such devices often gets lost and I suspect that it's because of too early DTR+RTS.

There is a workaround of doing stty -rtsdtr and then manually raising the signals (or just waiting for some FW timeout).
But, IMO, it would be nicer if the workaround wasn't needed.