tcp: Enable symmetric hashing by setting hash on outgoing conns

[gallatin]

Summary

Now that we can trust NICs to supply an identical hash result
to software, we can setup the inpcb hash on outgoing connections.
This gives us symmetric hashing, meaning packets should enter
and leave on the same NIC queue.

[adrian]

ok, so this looks fine, but I'm trying to remember where I used to assign the flowid/flowtype for a connection back in the day. On the mbuf side it used to be done in the driver side and then overriden by the network stack input path if it didn't match what was required (eg the magic required in ip reassembly, which is still there, but also to support completely software flow hashing.)

(remember way WAY back in the day the flowid/flowtype was also used by kip's flowdirector stuff for doing more explicit control over where flows went, and RSS just 'also' used it.)

[cc]

In D53104#1217628, @adrian wrote:

ok, so this looks fine, but I'm trying to remember where I used to assign the flowid/flowtype for a connection back in the day. On the mbuf side it used to be done in the driver side and then overriden by the network stack input path if it didn't match what was required (eg the magic required in ip reassembly, which is still there, but also to support completely software flow hashing.)

(remember way WAY back in the day the flowid/flowtype was also used by kip's flowdirector stuff for doing more explicit control over where flows went, and RSS just 'also' used it.)

+1 more usage is that SIFTR uses the TCP flowid for fast tracing. I am happy to see the flowid is present in default. :D

https://cgit.freebsd.org/src/tree/sys/netinet/siftr.c#n817

[zlei]

So typically a queue is bound to one core, then we have better cache localization for the same flow ? That sounds a good idea.

[adrian]

In D53104#1217970, @zlei wrote:

So typically a queue is bound to one core, then we have better cache localization for the same flow ? That sounds a good idea.

kinda, mostly, etc. that's the goal for RSS, but there were some hilariousness around whether the netisr threads are pinned to a CPU core or not.

I need to go back over the current code to see what it's currently doing and document its current and intended behaviours.

[cc]

sys/netinet/tcp_usrreq.c

1493–1494

If my eyes don't cheat, I think as we are sending out the packet, similar to CALC_FLOWID_OUTBOUND, the first input variable should be inp->inp_laddr as the src address, for example.

[cc]

sys/netinet/tcp_usrreq.c

1493–1494

My bad. This is different from fib4_calc_software_hash(). After verification in my code, I withdraw my comment above.