Page MenuHomeFreeBSD

replace cosqos field in pkthdr with a numa domain tag
ClosedPublic

Authored by gallatin on Apr 9 2019, 3:43 PM.

Details

Summary

Andre added a cosqos field to the pkthdr almost 6 years ago in r254804. This field is still unused by any in-tree consumers. I want to grab this field now for upcoming Numa work in the network stack.

I have a patchset that I'm working on which aligns many resources by numa domain, including inps, inpcb lb group, tcp pacing, lagg output link selection, etc. This work relies on being able to store the numa domain in the mbuf, so I want to grab this field for this use now before somebody else notices that it is unused and uses it for something else.

Note that the patchset which needs this cuts QPI traffic cross-domain traffic by roughly 50% (40% -> 20%). It also increase throughput by 35% on dual-socket Intel Cascade Lake based Xeons, and 75% on single-socket AMD Epyc. This is for a Netflix web workload, with all traffic being sent via sendfile().

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

gallatin created this revision.Apr 9 2019, 3:43 PM
gallatin edited the summary of this revision. (Show Details)Apr 9 2019, 3:46 PM
gallatin added a reviewer: jeff.
kib accepted this revision.Apr 9 2019, 3:48 PM
This revision is now accepted and ready to land.Apr 9 2019, 3:48 PM
markj accepted this revision.Apr 9 2019, 3:51 PM
This revision was automatically updated to reflect the committed changes.