I would like to understand how you are seeing or detecting the race?

We should have interrupts disabled in the legacy case.

For msi-x, I thought iflib modulated the queue through the IFDI.

What if we move the em_new_itr() call and /* Reset state */ stuff out of the em_msix_que() and into xxx_queue_intr_enable() before rearming IMS/EMIS?

Linux has a slightly different organization where the msi-x routine writes the previous lagging ITR, so the msi-x routine is only concerned with two values on the queue: itr_val and set_itr (a flag). The value that ends up in itr_val would be done by the equivalent of our interrupt filter. I don't know that this is better than the above suggestion (for one, why lag by one period?).

The meta point is I would like to avoid the long atomics or it starts eroding the benefits this is supposed to have. I think relying on the interrupt modulation for mutual exclusion and a compiler barrier for data coherence should be feasible, if you want to try to arrange it a particular way go ahead or if you want me to try, I'd just need to know how you are tracking the race so I know if I've fixed it.

atomic_load_long() has no special cost--it's a plain memory access. Its use here is to give read-once semantics to the access and signal to the reader that there might be some concurrent writes to that memory.

I claim that there's a race just be looking at where those fields are written. For instance, at the end of em_isc_txd_encap(), which is called in a threaded context, we have:

/* Sent data accounting for AIM */
txr->tx_bytes += pi->ipi_len;
++txr->tx_packets;

so what happens if those two fields are initially zero and this handler interrupts the thread before the increment of txr->tx_packets? The condition txr->tx_bytes && txr->tx_packets in the old code will be false, and bytes may be left uninitialized. In particular, the most important line in the diff is the addition of the assignment bytes = bytes_per_packet = 0;.

I don't think the new version of the code will be any more expensive, the rest of the diff just tries to clarify what's going on (at least to my taste). According to the bug report, it should be possible to reproduce this simply by booting a GENERIC-KMSAN kernel.

If you'd like to solve this a different way, please go ahead. The bug report came up because I asked the QAT driver developers to test with KMSAN and they hit this unrelated bug, so I wanted to unblock them.