We have like 32 reprobes that happen on boot for the USB sticks that the Netflix OCA hardware has for emergency backup booting. These race, as I've been able to get a reliable panic because we get into the state machine stuff twice due to the race presented here. This fixes things so that we only get into it once.
I added debugging and would see sequences like this:
```
(da0:umass-sim0:0:0:0): starting to reprobing...
(da0:umass-sim0:0:0:0): WP START 0xfffff800043ce000
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d5afe280
dastart() at dastart+0xb17/frame 0xfffffe00d5afe2d0
xpt_run_allocq() at xpt_run_allocq+0x1a2/frame 0xfffffe00d5afe320
daopen() at daopen+0x140/frame 0xfffffe00d5afe360
g_disk_access() at g_disk_access+0xef/frame 0xfffffe00d5afe3a0
g_access() at g_access+0x28f/frame 0xfffffe00d5afe410
g_access() at g_access+0x28f/frame 0xfffffe00d5afe480
g_eli_read_metadata_offset() at g_eli_read_metadata_offset+0xb5/frame 0xfffffe00d5afe540
g_eli_taste() at g_eli_taste+0xbb/frame 0xfffffe00d5afeb50
g_new_provider_event() at g_new_provider_event+0xaa/frame 0xfffffe00d5afeb70
g_run_events() at g_run_events+0x1a6/frame 0xfffffe00d5afebb0
fork_exit() at fork_exit+0x80/frame 0xfffffe00d5afebf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d5afebf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
(da0:umass-sim0:0:0:0): PROBEWP CCB state 18 da state 0 da flags 0x102e48 ccb 0xfffff800043ce000 softc 0xfffffe0384752000
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d5bc2ae0
dadone_probewp() at dadone_probewp+0x75/frame 0xfffffe00d5bc2b20
xpt_done_process() at xpt_done_process+0x358/frame 0xfffffe00d5bc2b60
xpt_done_td() at xpt_done_td+0xf5/frame 0xfffffe00d5bc2bb0
fork_exit() at fork_exit+0x80/frame 0xfffffe00d5bc2bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d5bc2bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
(da0:umass-sim0:0:0:0): WP START 0xfffff80006000000
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d5bc2a40
dastart() at dastart+0xb17/frame 0xfffffe00d5bc2a90
xpt_run_allocq() at xpt_run_allocq+0x1a2/frame 0xfffffe00d5bc2ae0
dadone_probewp() at dadone_probewp+0x126/frame 0xfffffe00d5bc2b20
xpt_done_process() at xpt_done_process+0x358/frame 0xfffffe00d5bc2b60
xpt_done_td() at xpt_done_td+0xf5/frame 0xfffffe00d5bc2bb0
fork_exit() at fork_exit+0x80/frame 0xfffffe00d5bc2bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d5bc2bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
(da0:umass-sim0:0:0:0): PROBEWP CCB state 18 da state 2 da flags 0x102e48 ccb 0xfffff80006000000 softc 0xfffffe0384752000
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d5bc2ae0
dadone_probewp() at dadone_probewp+0x75/frame 0xfffffe00d5bc2b20
xpt_done_process() at xpt_done_process+0x358/frame 0xfffffe00d5bc2b60
xpt_done_td() at xpt_done_td+0xf5/frame 0xfffffe00d5bc2bb0
fork_exit() at fork_exit+0x80/frame 0xfffffe00d5bc2bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00d5bc2bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
```
Notice that we get back into dadone_probewp() from a call to dadone_probewp(). Reversing the order of the state update and release fixed this race. I was hitting it every boot, now I rebooted 10 times w/o hitting it, or the stringent kasserts I added.
Note, the tracebacks are a little odd because dareprobe() is aggressively inlined. And the debug writes were added in the logical spots in the code (STARTs right before xpt_action, PROBE* lines right at the top of the dadone_probe* routines). dareprobe() prints the starting message... The funky 0x100000 bit in the da flags is a debug flag I added (but don't have in this patch) that I set when I took the reference, and cleared when I released it....