What happens is, when setting up an LACP lagg, we come through lacp_linkstate earlier via pr_addport, and it checks the interface's media status. We can't hold an rmlock while doing that with i.e. cxgbe
Example LOR:
begin_synchronized_op with the following non-sleepable locks held: exclusive rm if_lagg rmlock (if_lagg rmlock) r = 0 (0xfffff80186601a08) locked @ /d0/kev/freebsd/sys/net/if_lagg.c:703 stack backtrace: #0 0xffffffff8097d1d0 at witness_debugger+0x70 #1 0xffffffff8097e5fe at witness_warn+0x45e #2 0xffffffff8047d53c at begin_synchronized_op+0x3c #3 0xffffffff80488021 at cxgbe_media_status+0x31 #4 0xffffffff80a1d540 at ifmedia_ioctl+0x170 #5 0xffffffff804873c6 at cxgbe_ioctl+0x276 #6 0xffffffff8242542c at lacp_linkstate+0x4c #7 0xffffffff824259e1 at lacp_port_create+0x1f1 #8 0xffffffff82422b90 at lagg_ioctl+0x1340 #9 0xffffffff80a13a03 at ifioctl+0xe43 #10 0xffffffff80982db4 at kern_ioctl+0x2c4 #11 0xffffffff80982a6e at sys_ioctl+0x16e #12 0xffffffff80d25fc9 at amd64_syscall+0x549 #13 0xffffffff80d0715b at Xfast_syscall+0xfb