Page MenuHomeFreeBSD

ice(4): Fix traffic distribution on TX queues
Needs RevisionPublic

Authored by kgalazka on Fri, Mar 20, 9:40 AM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Apr 10, 8:40 PM
Unknown Object (File)
Fri, Apr 10, 7:15 AM
Unknown Object (File)
Wed, Apr 8, 9:57 AM
Unknown Object (File)
Tue, Apr 7, 1:00 PM
Unknown Object (File)
Sun, Apr 5, 7:07 PM
Unknown Object (File)
Wed, Apr 1, 5:26 PM
Unknown Object (File)
Sat, Mar 28, 12:06 PM
Unknown Object (File)
Fri, Mar 27, 1:05 AM
Subscribers

Details

Reviewers
erj
kbowling
glebius
gallatin
Group Reviewers
Intel Networking
Summary

Packets with no rsstype set were sent using single TX queue.
In case of multiple concurent UDP streams it heavly limitted
performance. Use mbuf hashing code factored out from LAGG driver
in commit <b7ba031ff71f2> to make sure that all configured
queues are used.

Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com>

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 71584
Build 68467: arc lint + arc unit

Event Timeline

gallatin requested changes to this revision.Sat, Apr 4, 9:57 PM

I hate that I'm saying this, but this isn't just an ice(4) problem... The same thing will happen for the queue selection in iflib itself for most other drivers, and for the queue selection in non-iflib drivers. So I think it should be solved in a more generic fashion.

I assume the packets are coming from an unconnected UDP socket transmit? Can you please solve this in a generic fashion in either ip / ip6 output or udp output itself by setting a hashtype there using rss_proto_software_hash_v[4|6] ?

This revision now requires changes to proceed.Sat, Apr 4, 9:57 PM

I'm testing that with GENERIC so with no RSS option. Without the patch both locally generated (iperf3) and forwarded (pkt-gen) UDP traffic is send using TX 0. I did a quick run on vanilla CURRENT with RSS enabled and it looks that in that case also only one TX queue is used, though it's not always queue 0. I haven't tried IPv6 yet.

As for other drivers I based this fix on MLX5. Some other drivers use curcpu % num_queues approach instead of hashing. We used that in pre-iflib IXGBE and IXL. I run some simple tests with that and I didn't see significant difference. I guess the results will depend on a use case. I was going to submit a patch for iflib after this gets more testing.