Fix panic when dequeuing nmdm tty from 'tty_list' after revoke(2).

[neel]

Summary

tty_rel_free() can be called more than once for the same tty so make sure
that the tty is dequeued from 'tty_list' only the first time.

The panic below was seen when a revoke(2) was issued on an nmdm device.
In this case there was also a thread that was blocked on a read(2) on the
device. The revoke would typically wake up the blocked thread and return
an error to userspace. In this case the reader also held the last reference
on the file descriptor so fdrop() ended up calling tty_rel_free() via
ttydev_close().

tty_rel_free() then tried to dequeue 'tp' again which led to the panic.

panic: Bad link elm 0xfffff80042602400 prev->next != elm
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f9c90460
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00f9c90510
vpanic() at vpanic+0x189/frame 0xfffffe00f9c90590
panic() at panic+0x43/frame 0xfffffe00f9c905f0
tty_rel_free() at tty_rel_free+0x29b/frame 0xfffffe00f9c90640
ttydev_close() at ttydev_close+0x1f9/frame 0xfffffe00f9c90690
devfs_close() at devfs_close+0x298/frame 0xfffffe00f9c90720
VOP_CLOSE_APV() at VOP_CLOSE_APV+0x13c/frame 0xfffffe00f9c90770
vn_close() at vn_close+0x194/frame 0xfffffe00f9c90810
vn_closefile() at vn_closefile+0x48/frame 0xfffffe00f9c90890
devfs_close_f() at devfs_close_f+0x2c/frame 0xfffffe00f9c908c0
_fdrop() at _fdrop+0x29/frame 0xfffffe00f9c908e0
sys_read() at sys_read+0x63/frame 0xfffffe00f9c90980
amd64_syscall() at amd64_syscall+0x2b3/frame 0xfffffe00f9c90ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00f9c90ab0

• syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b78d8a, rsp = 0x7fffffbfdaf8, rbp = 0x7fffffbfdb30 ---

Test Plan

Write a program that opens an nmdm device, creates a thread that blocks on
a read of this device and then closes the device. While this program is
blocked on the read() the nmdm device is revoked.

Since this is a race the scenario above should be retried in a loop and
without the patch it will lead to the panic in tty_rel_free().

[neel]

C program to open nmdm device, create a thread that blocks on a read() of the device and then closes it.

nmdmclosetest.c1 KBDownload

Shell script that uses the compiled output of the program above to reproduce the panic.

nmdm_revoke.sh279 BDownload

[glebius]

I'm not super confident here, but the change looks correct to me.

I tried to add Ed Shouten as reviewer, who is author of the code, but failed.

[neel]

Thanks for the review.

I feel a bit more confident about this because it can be consistently reproduced without the patch. After the patch the script can run overnight without any panic.

ed@ is not on Phabricator so I have sent him an email offline asking for a code review.

[ed]

Looks good! Be sure to commit!