Page MenuHomeFreeBSD

teach ena driver about RSS kernel option
ClosedPublic

Authored by avg on May 6 2020, 8:45 AM.

Details

Summary

Networking is broken if the driver configures its (virtual) hardware to use
a hash algorithm (or a key) different from the one that the network stack
(software RSS) uses. This can be seen with connections initiated from the
host. The PCB will be placed into the hash table based on the hash value
calculated by the software. The hardware-calculated hash value in reponse
packets will be different, so the PCB won't be found.

The work is sponsored by Panzura.

Test Plan

Tested with a kernel compiled with 'options RSS' on an instance with ena
driver.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

avg requested review of this revision.May 6 2020, 8:45 AM

update the module makefile

This actually looks good to me!

Sigh - this is why when I added the flowid we didn't set the hash for TCP - we relied on the result from the hardware ...

Sigh - this is why when I added the flowid we didn't set the hash for TCP - we relied on the result from the hardware ...

That's like 80% of the problem, the other 20% was getting the pcb into the right group early on and not have to go move it between the RSS buckets.

The flowid stuff is separate from RSS per se. If you use flowid without the microsoft style RSS with the per-CPU work distribution then yeah, you don't need the software hashing. But if you want to have it partition up the work early that also includes putting the pcb into the right RSS hashed pcbgroup, and that requires a software hash.

It's only done, like, once..

Sigh - this is why when I added the flowid we didn't set the hash for TCP - we relied on the result from the hardware ...

That's like 80% of the problem, the other 20% was getting the pcb into the right group early on and not have to go move it between the RSS buckets.

The flowid stuff is separate from RSS per se. If you use flowid without the microsoft style RSS with the per-CPU work distribution then yeah, you don't need the software hashing. But if you want to have it partition up the work early that also includes putting the pcb into the right RSS hashed pcbgroup, and that requires a software hash.

It's only done, like, once..

Was the PCBGROUPS stuff actually done? IIRC it relied on netisr changes which were incomplete. I got substantially worse perf turning it on for a number of reasons.

Hi, sorry for late response, missed that previously. I let know about the change to Amazon folks so that they can review it internally as well. I'll update.

We managed to get an ack from Amazon, as well.

This revision is now accepted and ready to land.Jun 23 2020, 12:33 AM
This revision was automatically updated to reflect the committed changes.