Page MenuHomeFreeBSD

Enable multipath for outbound connections
ClosedPublic

Authored by melifaro on Sep 22 2020, 7:47 PM.

Details

Summary

Implement flowid calculation for outbound connections to enable splitting connections over multiple paths.

Multipath routing relies on mbuf flowid data for both transit and outbound traffic.
Current code fills mbuf flowid from inp_flowid for connection-oriented sockets.
However, inp_flowid is currently not calculated for outbound connections.

This diff creates simple hashing functions and starts calculating hashes for TCP,UDP/UDP-Lite and raw IP if multipath routes are present in the system.

RSS and ROUTE_MPATH

Multipath routing currently does not require any special properties from the hashing function, except good flow distribution.
RSS for outbound connections (which is actually XPS) require hashing function to be compatible with card hashing to achieve optimal TX ring assignment.

RSS requires PCBGROUP, which makes coupling RSS and ROUTE_MPATH by requiring RSS inconvenient.

With that in mind,

  • use RSS-derived hashing functions for outbound connections when RSS in enabled and
  • use light-weight ROUTE_MPATH hashing functions otherwise
  • implement a simple wrapper layer (see net/route.h) to switch between the implementation without touching callers.
NOTE: I intend to commit this change on Sunday, October 16

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

It seems you missed hash calculation for udp[6]_output(), when socket isn't connected and destination address is specified by caller.

This revision is now accepted and ready to land.Sep 23 2020, 9:09 PM
emaste added inline comments.
sys/net/route/mpath_ctl.c
72 ↗(On Diff #77373)

Where does this come from?

sys/net/route/mpath_ctl.c
72 ↗(On Diff #77373)

That's the default key from Microsoft RSS spec, copied from rss_config.c

Fix raw IPv6, add connectionless UDP.
Fix tests.

This revision now requires review to proceed.Wed, Oct 7, 7:45 PM
sys/netinet/udp_usrreq.c
1480 ↗(On Diff #77988)

It seems you forgot #include "opt_route.h" and this code was not built. :)
Note, a few lines below we have code that sets flowid and flowtype.

sys/netinet6/udp6_usrreq.c
79 ↗(On Diff #77988)

Also opt_route.h is missing.

968 ↗(On Diff #77988)

Probably we don't need to calculate flowid twice when both ROUTE_MPATH and RSS are defined.

Improve ROUTE_MPATH & RSS configurations.

melifaro marked an inline comment as done.

Address ae@ comment on udp6_usrreq.

LGTM. However it would be nice, if you consider my comments :)

sys/net/route.h
136 ↗(On Diff #78337)

I think it would be better to name these all with upper case, to have understanding from the code that uses them that we use macro. Or maybe move all hashing related code to the same place, and do conditional build there.

sys/net/route/mpath_ctl.c
68 ↗(On Diff #78337)

Maybe we should allow set it from loader.conf? I.e. use CTLFLAG_RDTUN flag?

This revision was not accepted when it landed; it landed in state Needs Review.Sun, Oct 18, 5:16 PM
This revision was automatically updated to reflect the committed changes.