nvme: Reduce traffic to the doorbell register

[imp]

Summary

Reduce traffic to doorbell register when processing multiple completion
events at once. Only write it at the end of the loop after we've
processed everything (assuming we found at least one completion).

Sponsored by: Netflix

Test Plan

"boots for me"

[mav]

sys/dev/nvme/nvme_qpair.c

669

It seems cq_head can be incremented above without done incremented and without panic on non-debug kernel, in which case the register won't be written, while I think it should unless we want to get back here again.

[imp]

sys/dev/nvme/nvme_qpair.c

635–636

I'll move this up a few lines.... though I'm thinking of making this a 'panic', but maybe not yet.

[imp]

If we 'accept' it, then bump the count of items we've cleared.

[imp]

sys/dev/nvme/nvme_qpair.c

669

Good catch. Fixed.