Let's try the benefit of this patch on a 8-core Atom with a 10G Chelsio, during a forwarding bench with tcdpump running:

x r347519: inet4 smallest size packets-per-second forwarded with tcpdump running
+ r347519 and D20224: inet4 smallest size packets-per-second forwarded with tcpdump running
+--------------------------------------------------------------------------+
|   xx              x                                          +  +   +  ++|
||___M__A______|                                                           |
|                                                                |___AM___||
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       3682162       3816554       3684703     3711296.2     58889.489
+   5     4179733.5       4274492       4240164     4234216.7     39505.425
Difference at 95.0% confidence
        522920 +/- 73130.8
        14.09% +/- 2.16578%
        (Student's t, pooled s = 50143)

Applying ngie's comments

I'm generating about 2000 flows and I'm seeing a big improvement by enabling tx_abduction too with iflib.
But my understanding is this review is not about tx_abduction but about TX_BATCH_SIZE, and that I had to check if this TX_BATCH_SIZE patch will bring better performances in forwarding mode.
This is why I've made 2 DoS forwarding benches:

• One comparing TX_BATCH_SIZE patch against 2 head configured identically
• One comparing performance impact brings by this TX_BATCH_SIZE patch against the performance brings by tx_abduction

It's the standard "DoS" method: I'm unidirectional sending line-rate of smallest size packet.

Ok, so let's try again with this latest version (I'm calling this one D18532v3):

Here are my classical DoS benches with ipfw (stateless with one rule "allow ip from any to any") results on 3 different setup:

Hi,
I've got igb only NIC, not em, so I can't use my lab to bench this new review version.

Other platform (supermicro 5018A-FTN4, Atom C2758 8-cores at 2.4Ghz with intel 82599ES 10-Gigabit):

x r342020 with D18532(old: diff51924) and tx_abdicate disabled (default): inet4 pps
+ r342020 with D18532(new: diff51986) and tx_abdicate disabled (default): inet4 pps
+--------------------------------------------------------------------------+
| x             xx                      +    +          *               + +|
||______________M____A____________________|                                |
|                                         |_____________MA_______________| |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       3179260       3260787       3200772     3208284.5     30758.531
+   5       3235848       3287655       3260728     3262531.6     23384.011
Difference at 95.0% confidence
        54247.1 +/- 39846.4
        1.69084% +/- 1.25533%
        (Student's t, pooled s = 27321.2)

Comparing DoS benches results with the previous Diff version applied and this one didn't show difference.

Here is my DoS benches results on my smallest 2 hardwares.
First serie comparing the benefit of D18532, so with tx_abdicate disabled (default behaviour)

Benches results using Denial-of-Service mode (line-rate at maximum pps) on small hardware with pf+pfysnc (using loopback interface):
PC Engines APU 2 (4 cores AMD GX 1Ghz):

Yes, I confirm a little improvement on small devices (same iflib.tx_abdicate=1 and ip_redirect disabled).

I've forgot to disable IP redirect in my previous bench.
Here are the other results, on 3 different small platforms (-5% on all).
All of these are using common tuning features for routing/firewalling which are:

• net.inet.ip.redirect=0 and net.inet6.ip6.redirect=0, this allow to re-enable fastforwarding path
• dev.igb|ix.X.iflib.tx_abdicate=1, because they are receiving maximum link packet rate

Here is the first result on forwarding performance impact on low-end hardware (AMD GX-412TC, 4Cores and Intel i210AT NIC) :

In D15789#334385, @bz wrote:

I am totally not surprised by these numbers. However (a) did you do the same test for IPv6? (b) is that a forwarding setup or an end node setup? (c) how many route updates per second did you try on a forwarding node?

On a 2 sockets, 12 Core Xeon E5 2650 with a Mellanox ConnectX-4:

Same improvement on 8-cores ATOM with ixgbe drivers:

Lot's better on 4core AMD, but need to reset my non-IPMI Netgate tomorrow on the lab:

Panic on PC engines APU2 (AMD GX-412TC):