Page MenuHomeFreeBSD

iflib: add a simple transmit routine
ClosedPublic

Authored by gallatin on Aug 14 2025, 9:31 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Oct 10, 6:09 PM
Unknown Object (File)
Fri, Oct 10, 6:09 PM
Unknown Object (File)
Fri, Oct 10, 6:09 PM
Unknown Object (File)
Fri, Oct 10, 1:11 PM
Unknown Object (File)
Wed, Oct 8, 9:20 PM
Unknown Object (File)
Thu, Oct 2, 6:05 AM
Unknown Object (File)
Wed, Sep 24, 7:30 AM
Unknown Object (File)
Sun, Sep 21, 3:02 PM
Subscribers

Details

Summary

While mp_ring can provide amazing scalability in scenarios where the number of cores exceeds the number of NIC tx rings, it can also lead to greatly reduced performance in simpler, high packet rate scenarios due to extra CPU cycles and cache misses stemming from its complexity.

In testing on a 400GbE NIC in an AMD 7502P EPYC server, this simple tx routine is roughly 2.5 times as fast as mp_ring (8Gbs -> 20Gb/s). and 5x as fast as mp_ring with tx_abdicate=1 (4Gbs) for a simple in-kernel packet generator, which is closed source currently. It also shows 50% speedup for a simple netperf -tUDP_STREAM test (5Gb/s -> 8Gbs).

This change is mostly a noop, as it not enabled by default. The one exception is the change to iflib_encap() to immediately reclaim completed tx descriptors, and only failing the transmit & scheduling a later reclaim if iflib_completed_tx_reclaim() didn't free enough descriptors.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

LGTM, it's out of the way aside from a branch and the change to iflib_encap (is this change strictly necessary for simple or can it be committed separately?).

I don't know that we have a great forum for this stuff anymore but it would be interesting to work through some kind of roadmap and see if we want to converge on simple or mp_ring over time.

This revision is now accepted and ready to land.Aug 18 2025, 10:55 PM
vmaffione added inline comments.
FreeBSD/share/man/man4/iflib.4
77 ↗(On Diff #160371)

s/igored/ignored

This revision was automatically updated to reflect the committed changes.