aq(4): Fix RSS indirection table OOB write and queue distribution

[nick_spun.io]

Summary

Two related fixes to aq(4)'s RSS indirection table handling:

1. Fix an out-of-bounds stack write in aq_hw_rss_set(). RSS table entries are 3 bits (8 queues max), but with more than 8 RX rings rss_table[] holds larger values; the 32-bit write then spills one uint16_t past bitary[] and corrupts the stack, so the NIC never links or the kernel panics. Mask each value to 3 bits and pack 16 bits at a time to keep the write in bounds.

2. Build the indirection table in aq_if_attach_post() with a modulo over min(rx_rings_count, HW_ATL_RSS_INDIRECTION_QUEUES_MAX) instead of i & (rx_rings_count - 1), which assumed a power-of-two ring count.

Test Plan

Confirmed on AQC107 (firmware 3.1.88) under FreeBSD 15.0-RELEASE: with the OOB write the interface attached but stayed "no carrier"; with the fix the link negotiates 1000baseT full-duplex and passes traffic.

[nick_spun.io]

This comment was removed by nick_spun.io.

[nick_spun.io]

This comment was removed by nick_spun.io.

[nick_spun.io]

This comment was removed by nick_spun.io.

[nick_spun.io]

Think this is in a pretty reasonable state now.

[nick_spun.io]

sys/dev/qlxge/qls_hw.c:1004 also seems to use the incorrect RSS queue pattern - may be worth specifically mentioning that anti-pattern in the iflib man page or in the rss comments somewhere

[adrian]

oh, hm, how's 8 RSS queues but > 8 RX queues work? In any case, good catch!

[nick_spun.io]

The default pattern is to repeat the RSS queues across the RX queues so if there are 4 RSS queues but 8 RX queues you end up with 1 2 3 4 1 2 3 4 but this is actually configurable at runtime for traffic steering and whatnot

There are actually 64 RX queues on this card (traffic lands in them via hashing) so even with the maximum of 8 RSS queues you still end up duplicating them several times