mpi3mr: Reduce the scope of the reset_mutext

[imp]

Summary

Reduce the scope of reset_mutext to protect the msleep in the watch dog
thread as well as the MPI3MR_FLAGS_SHUTDOWN bit. Use it to protect the
wakeup in mpi3mr_detach so this thread can exit sooner when we're trying
to do an orderly shutdown.

It's an open question if this should protect sc->unrecoverable, and if
we should wakeup the watchdog thread when we set it. We might also want
to move too booleans for the three flags that we have now in
mpi3mr_flags. There are a number of U8s that should really be bools and we
might want to also group them together to pack softc better.

[mav]

The lock seems should protect not only msleep(), that is indeed pointless, but also both MPI3MR_FLAGS_SHUTDOWN accesses and also wakeup(&sc->watchdog_chan) in mpi3mr_pci_detach(). Otherwise the thread may not exit until the timeout expiration, that is a waste of time.

[imp]

In D42539#972011, @mav wrote:

The lock seems should protect not only msleep(), that is indeed pointless, but also both MPI3MR_FLAGS_SHUTDOWN accesses and also wakeup(&sc->watchdog_chan) in mpi3mr_pci_detach(). Otherwise the thread may not exit until the timeout expiration, that is a waste of time.

I agree. I hadn't had the time to chase this issue to ground.

Do we want to do a wakeup when unrecoverable is set to 1 too? I'm thinking no because the watchdog thread exiting quickly doesn't buy us much. But if so, then we should lock around setting it and issue a wakeup. It's done so many places, an inline function likely should be written.... I don't understand how things wind down once this is set enough to know if there's a benefit or not.

[imp]

implement mav's idea about wakeup

[mav]

sys/dev/mpi3mr/mpi3mr.c

3039–3040

We should check the same condition as below before the msleep(), so we would not enter the sleep while shutdown is already requested.

[imp]

Better flow

[mav]

This version still may go to sleep with MPI3MR_FLAGS_SHUTDOWN flag set, that is suboptimal.

[imp]

this time with the right change and to the right change.