nvme: Add exclusion for ISR

[imp]

Summary

Add a basically uncontended spinlock that we take out while the ISR is
running. This has two effects: First, when we get a timeout, we can
safely call the nvme_qpair_process_completions w/o racing any ISRs.
Second, we can use it to ensure that we don't reset the card while
the ISRs are active (right now we just sleep and hope for the best,
which usually is fine, but not always).

Sponsored by: Netflix

[gallatin]

Don't you need a mtx_init() ? Or did I just miss it..?

sys/dev/nvme/nvme_private.h

204

struct mtx_padalign does what you want

[imp]

update, per drew's review

[imp]

update per jhb's observation that my locking was screwed up

[chuck]

sys/dev/nvme/nvme_ctrlr.c

1757

Is the call to nvme_pre_reset() needed here? Both paths to fail occur after the initial call to nvme_pre_reset()

sys/dev/nvme/nvme_qpair.c

551

New-line?

[imp]

sys/dev/nvme/nvme_ctrlr.c

1757

This needs to be nvme_post_reset() since we called nvme_pre_reset() before jumping to fail.

sys/dev/nvme/nvme_qpair.c

551

Oh yea. other code was there briefly during development, but it's gone now.

[imp]

update per chuck's feedback