{F166942590}Port buf_ring/drbr deferred transmit from mxge(4) to iflib. This
makes simple_tx outperform mp_ring by a wide margin (1.9x to 9x) at
all concurrency (fan-in) levels in my testing. This also adds ALTQ
support to simple_tx and gets us one step closer to removing mp_ring.
The drbr support has 2 improvements from mxge(4):

1. Limit the number of producers attempting to defer traffic and block on ift_mtx rather than having many producers spinning on br_prod_tail. Producers are bounded via net.iflib.simple_max_producers, which defaults to 1 & which measured best by a fairly wide margin in testing using a home-grown packet generator sending traffic from N threads to a single output queue on a 96c/192t AMD Turin. Traditional unlimited deferrals can be achieved by setting it larger than ncpu, and setting it to 0 falls back to the current behavior of always grabbing a mutex.

2. Block rather than drop when the buf ring fills. This limits useless work done by producers and again limits producers spinning on br_prod_tail when simple_max_producers is large.

The drbr buf_ring helper functions have been modified to support
blocking by removing the mbuf free in drbr_enqueue to
add a drbr_enqueue_nodrop() variant.

Drain from the tx task is bounded by net.iflib.simple_drain_quota
(default 8) so it cannot starve rx on a shared taskqueue or ithread.
The tx task is also modified to use a trylock and not block
waiting for another thread draining the buf ring.

We force queue 0 when ALTQ is enabled, as the mp_ring path does.

Performance data below from a 96c Turin, using a 400g bnxt interface, with my pkgen
sending 1342B UDP packets with separate headers (so as to be realistic and
give encap work to do). A single txq is selected by forcing the flowid in
the pkt generator., Gb/s at 1/2/4/8/16/32/64 threads per
queue. Data for this change is taken as the worst of three runs:

mp_ring      20 44 48 35 14  7  4
simple_tx    44 44 44 43 40 36 32
+this change 44 90 90 86 57 50 35

The only remaining feature to support is ice(4)'s special tx queue
selection. After that, I plan to make simple_tx the default
and then remove mp_ring after a month or two and rename all
the simple_tx* to iflib*