Excellent news, and it looks like there's still some performance left on the table there. I guess the question now is if I consider the theory that getting rid of the mp ring improves performance proven and start gutting it, or if I finish separating the TX path from the reclaim path and clean this hack up more so further testing can be done.

So two things jump out at me here... first, a quick look suggests _task_fn_tx() -> iflib_txq_drain() is recursing on the mutex, and second the interrupt handler is contending with tx for no good reason.

Create a iflib_txq_drain_bypass() function called only from
iflib_if_transmit_bypass() which doesn't have the loop,
volatile/devolatile casts, conditional prefetching, txq-cast-to-mbuf,
etc overhead and only locks/unlocks once.

I assume the two piles in the bypass flamegraphs which don't start at fork_exit() are just because the stack sample isn't deep enough to correlate them over and that the iflib_txq_drain() samples in the wide peak are actually the ones that are directly in iflib_txq_drain?

For the bypass disabled results, is tx_abdicate set?

Use a tunable instead of a sysctl to toggle mp_ring bypass.

Wait... why is iflib_if_transmit() being called in both? It should be using iflib_if_transmit_bypass() instead.

Cannot reproduce with D20834 appplied.

Ping.

Ok, I didn't think these functions were hot enough to be worth optimizing, and I'm not sure anything can be optimized around E1000_WRITE_*

Phew, great job. I've always been a bit uneasy about the MSI/INTx stuff, but this seems to have cleared everything up.

Makes sense.

Use rtsdtr as the stty argument to set the default mode, and -rtsdtr to
disable automatically asserting them on open().

Address feedback, add support to umcs.

"Reference" count core offsets and free() when all drivers unloaded.

Fix style(), clean up default sysctl values, log message on malloc() failure.

In D20031#430635, @kevans wrote:

Tagging @ian as well, as he may be interested in this for $work purposes.

I think this also needs to cover the DTR|RTS setting in ucom_open (^/sys/dev/usb/serial/usb_serial.c) for completeness sake, but I defer to @imp and @ian as they've likely got better intuition of how that should work.

I'd really like to see this part split out into a separate review and committed soon, because it shouldn't see any contention- it adds control that is desirable to the people that know they need it, and doesn't really affect anyone else.

Looks good to me.

And again now that I've joined iflib...

Accepted. (derp)

LGTM

style(9) nits.

FILTER_SCHEDULE_THREAD is a flag, not a discrete value. If it's set, schedule the
gtask and return FILTER_HANDLED.

Return FILTER_HANDLED rather than FILTER_SCHEDULE_THREAD when gtask (potentially) scheduled
per @waddlesplash_gmail.com.

Some bits of the upstream patch didn't paply cleanly... added manually.

Put the result variable back in. *facepalm*

Unpack the conditional for easier reading.

Remove unrelated stuff.

Looks good to me.

I love everything about this. If you plan to use the summary as the commit message though you should modify "change the M_NOWAIT from malloc(9) calls into M_NOWAIT." bit to mention M_WAITOK.

Looks good. This may also fix where bar == -1 && pci_msix_count(dev) == 0 (or break it less... or something)

I think that the inability to map an MSI-X table should likely not be bootverbose... this is an allocation failure which will significantly impact device performance. While the user should know that MSI-X is disabled, I'm not sure the user should be expected to know that the allocation will fail.

No objections, though I do agree with cem@ in principle regarding putting it in the GENERICs. When 3rd-party drivers start relying on it, it would violate POLA removing Intel NICs from the kernel would break a 3rd-party driver. For now it's likely "fine".

So the basic goal is to have the test at sys/net/if_ethersubr.c:583 see a consistent state of the changes from the /* Change the interface type */ line to the lagg_proto_addport() line.

RLOCK() before setting ifp->if_type

In D18761#400738, @pkelsey wrote:

In D18761#400698, @shurd wrote:

My only concern here is making IFLIB_MAX_RX_SEGS available to the driver...

In the current iflib implementation, the reality of course is that IFLIB_MAX_RX_SEGS is the upper limit of what a driver can use. My approach is to just give it as much runway as the current iflib implementation allows.

My only concern here is making IFLIB_MAX_RX_SEGS available to the driver...

Looks good, only a tiny nitpick on the fail label.

In D18532#399851, @olivier wrote:

Hi,
I've got igb only NIC, not em, so I can't use my lab to bench this new review version.

In addition to trying to keep the TXQ full, use a mp_ring size that's half the
number of descriptors. Previously, the mp_ring was a fixed size which happened
to be twice the default size of the txq for my em devices.

In D18545#395423, @jacob.e.keller_intel.com wrote:

I'm not entirely certain how to reproduce the original setup that triggered the bug, so some help with that would be appreciated.