Page MenuHomeFreeBSD

make lacp's use_numa hashing aware of send tags
ClosedPublic

Authored by gallatin on Feb 23 2020, 9:27 PM.
Tags
None
Referenced Files
F136021692: D23811.id.diff
Sat, Nov 15, 4:10 AM
F135930689: D23811.id68725.diff
Fri, Nov 14, 6:46 AM
Unknown Object (File)
Thu, Nov 13, 7:42 PM
Unknown Object (File)
Tue, Nov 11, 10:50 PM
Unknown Object (File)
Mon, Oct 27, 5:48 PM
Unknown Object (File)
Mon, Oct 27, 5:48 PM
Unknown Object (File)
Mon, Oct 27, 12:19 PM
Unknown Object (File)
Tue, Oct 21, 5:08 AM
Subscribers

Details

Reviewers
hselasky
jhb
rrs
Group Reviewers
transport
Summary

When I did the use_numa support, I missed the fact that there is a separate hash function for send tag nic selection. So when use_numa is enabled, ktls offload does not work properly, as it dos not reliably allocate a send tag on the proper egress nic since different egress nics are selected for send-tag allocation and packet transmit. To fix this, this change:

  • refectors lacp_select_tx_port_by_hash() and lacp_select_tx_port() to make lacp_select_tx_port_by_hash() always called by lacp_select_tx_port()
  • pre-shifts flowids to convert them to hashes when calling lacp_select_tx_port_by_hash()
  • add a numa_domain field to if_snd_tag_alloc_params
  • plumb the numa domain into places where we allocate send tags

In testing with NIC TLS setup on a NUMA machine, I see thousands of output errors before the patch when enabling kern.ipc.tls.ifnet.permitted=1. After the change, I see no errors, and I see the NIC sysctl counters showing active TLS offload sessions.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 29561