netinet: Generate a random RSS key on boot.

[nc]

Summary

netinet: Generate a random RSS key on boot.

Submitted by: Neel Chauhan <neel AT neelc DOT org>

[debdrup]

Did you mean to add some reviewers for this too? :)
Looks like @rwatson might have some thoughts on this :)

[nc]

Sure.

[melifaro]

sys/net/rss_config.c

251

Are we sure that there is enough entropy available at the moment of calling it?

[nc]

sys/net/rss_config.c

251

I'm not sure, I'm not an expert on entropy, but we could wait for entropy before calling this.

[kevans]

sys/net/rss_config.c

251

When in doubt, perhaps CC csprng

[nc]

@kevans thanks for the suggestion, I added csprng to this thread.

[cem]

sys/net/rss_config.c

251

With csprng hat on: The random subsystem does not guarantee it has been seeded at any point during auto-configuration (SYSINIT).

If it has not been seeded yet, arc4random currently gives you a weakly random key (hash of the CPU clock cycle counter, essentially); that is still marginally more random than the hardcoded value this revision deletes, but not especially random. The API doesn't tell you which kind of output you got.

As far as waiting for entropy before initializing the key, doesn't that break RSS for existing connections? If that is tolerable, then that seems like a good solution. I'd want someone familiar with networking to approve that design (and this design!).

I'm not sure what the deleted XXXRW comment is referring to as far as rss_isbadkey().

As far as the actual arc4random_buf invocation, s/&rss_key/rss_key/.

[nc]

@cem thanks for the comments.

I fixed arc4random_buf().

About waiting for entropy, I think you're right.

[markm]

sys/net/rss_config.c

251

This scares me less than the previously hard-coded key, but if we are going to the trouble to randomise, then we need to sequence it properly.

[avg]

I have a vague memory, maybe wrong, that commonly used fixed RSS keys were selected because they had some property (-ies).
So, maybe just being random is not good enough?
I think that hypothetical rss_isbadkey was mentioned for a reason?

[ae]

In D24989#552576, @avg wrote:

I have a vague memory, maybe wrong, that commonly used fixed RSS keys were selected because they had some property (-ies).
So, maybe just being random is not good enough?
I think that hypothetical rss_isbadkey was mentioned for a reason?

I also have such feeling. For example, you have some server that handles some serious workload, but after reboot due to the new key it will not be able to handle the same workload.

[nc]

I believe Linux just uses random keys:

• https://github.com/torvalds/linux/blob/29d9f30d4ce6c7a38745a54a8cddface10013490/net/ethtool/ioctl.c#L924

Callers (example):

• https://github.com/torvalds/linux/blob/29d9f30d4ce6c7a38745a54a8cddface10013490/drivers/net/ethernet/emulex/benet/be_main.c#L3728
• https://github.com/torvalds/linux/blob/29d9f30d4ce6c7a38745a54a8cddface10013490/drivers/net/ethernet/emulex/benet/be_main.c#L3728

However, DPDK uses something called "MAXIMALLY EQUIDISTRIBUTEDCOMBINED LFSR GENERATORS" (paper: https://www.ams.org/journals/mcom/1999-68-225/S0025-5718-99-01039-X/S0025-5718-99-01039-X.pdf):

• https://github.com/DPDK/dpdk/blob/905a0c208267806f5e1f2021accec4302681e4a0/lib/librte_eal/common/rte_random.c#L131
• Calls https://github.com/DPDK/dpdk/blob/905a0c208267806f5e1f2021accec4302681e4a0/lib/librte_eal/common/rte_random.c#L101

Caller (example):

• https://github.com/DPDK/dpdk/blob/c3d30e9d753ef3121c25b9403bf56ee09f2a2086/drivers/net/bnx2x/bnx2x.c#L6701

[nc]

In D24989#552576, @avg wrote:

I have a vague memory, maybe wrong, that commonly used fixed RSS keys were selected because they had some property (-ies).
So, maybe just being random is not good enough?
I think that hypothetical rss_isbadkey was mentioned for a reason?

I Google searched this and haven't found much evidence on RSS keys. I could be wrong as well.

EDIT: There is evidence on "Symmetric" RSS keys like seen here: http://galsagie.github.io/2015/02/26/dpdk-tips-1/

Is that what you're talking about?

[nc]

Should I sequence the key into "symmetrical RSS" where the first 32 bits == the next 32 bits, and all subsequent 16 bits chunks are equal?. Or is it something else?

[avg]

In D24989#552693, @neel_neelc.org wrote:

I believe Linux just uses random keys:

• https://github.com/torvalds/linux/blob/29d9f30d4ce6c7a38745a54a8cddface10013490/net/ethtool/ioctl.c#L924

You are right. So, my concern was invalid.

[adrian]

i remember there was some concern in the past where there were very bad rss key choices out there. is there a reason for actually pushing for a random rss key?

I stuck with the microsoft rss key (and a symmetric rss key at norse) specifically so there wouldn't be boot to boot variation in traffic patterns when doing testing/evaluation.

[damien.deville_stormshield.eu]

Hi everyone,

At stormshield we are using a similar hand-made patch so i can give you some feedback about this feature.

We support symmetric and non-symmetric static or random rss key, using the following systctls:

• net.inet.rss.random
• net.inet.rss.symmetric (using a 16bit pattern)
• net.inet.rss.symmetric_byte0 + net.inet.rss.symmetric_byte1 (to force the 16bits pattern to make the hash symmetric)

We were also forced to add a reseed sysctl proc after noticing that the initial entropy is too low during rss key init, and we call it after boot just before loading our network kernel modules. Before this we had some product that were using RSS key that fail to provide proper distribution of packets. This change was made before we start using the initial entropy feature of the loader so i am not sure if it is still relevant.

If some of you have interest i can share our patch (in private) which only support the rss_getkey() API and not all the other rss_xxx functions has we are not using them in our codebase.

Damien for Stormshield.