Page MenuHomeFreeBSD

ena: Fix misconfiguration when requesting regular LLQ
ClosedPublic

Authored by akiyano_amazon.com on Fri, Apr 25, 10:38 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, May 7, 5:05 AM
Unknown Object (File)
Mon, May 5, 11:49 PM
Unknown Object (File)
Mon, May 5, 7:52 AM
Unknown Object (File)
Fri, May 2, 6:00 AM
Unknown Object (File)
Thu, May 1, 12:56 AM
Unknown Object (File)
Tue, Apr 29, 9:59 PM
Unknown Object (File)
Tue, Apr 29, 11:01 AM
Unknown Object (File)
Tue, Apr 29, 6:59 AM
Subscribers

Details

Summary

Patch 0a33c047a443 introduced new values to
hw.ena.force_large_llq_header. The default value of 2 means no
preference, while 0 and 1 act as the previous false and true
respectively, which allowed forcefully setting regular or large LLQ.

There are 2 ways to force the driver to select regular LLQ:

  1. Setting hw.ena.force_large_llq_header = 0 via sysctl.
  2. Turning on ena express, which makes the recommendation by the FW to be regular LLQ.

When the device supports large LLQ but the driver is forced to
regular LLQ, llq_config->llq_ring_entry_size_value is never initialized
and since it is a variable allocated on the stack, it stays garbage.

Since this variable is involved in calculating max_entries_in_tx_burst,
it could cause the maximum burst size to be zero. This causes the driver
to ignore the real maximum burst size of the device, leading to driver
resets in devices that have a maximum burst size (Nitro v4 and on. see
[1] for more information).

In case the garbage value is 0, the calculation of
max_entries_in_tx_burst divides by 0 and causes kernel panic.

The patch modifies the logic to take into account all use-cases and
ensure that the relevant fields are properly initialized.

[1]: https://docs.aws.amazon.com/ec2/latest/instancetypes/ec2-nitro-instances.html

Fixes: 0a33c047a443 ("ena: Support LLQ entry size recommendation from device")
Approved by: cperciva (mentor)
MFC after: 3 days
Sponsored by: Amazon, Inc.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

akiyano_amazon.com created this object with edit policy "No One".
akiyano_amazon.com changed the edit policy from "No One" to "All Users".Sun, Apr 27, 5:41 AM
akiyano_amazon.com edited the summary of this revision. (Show Details)

Fixed comments by cperciva@ that I got via email since I blocked accidentally comments here

llq_config->llq_num_decs_before_header =
  • ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2;

+ ENA_ADMIN_LLQ_NUM_DESCS_BEFORE_HEADER_2;

No reason for this whitespace change? The original 4 spaces is style(9) conformant since this is a long-line-wrapping indent.

+ /* If the user didn't specify regular LLQ explicitly */
+ use_large_llq = (ena_force_large_llq_header !=
+ ENA_LLQ_HEADER_SIZE_POLICY_REGULAR);
+
+ /* The device supports large LLQ */
+ use_large_llq &= !!(llq->entry_size_ctrl_supported &
+ ENA_ADMIN_LIST_ENTRY_SIZE_256B);
+
+ /* If the user set no preference, check if the device recommends */
+ if (ena_force_large_llq_header == ENA_LLQ_HEADER_SIZE_POLICY_DEFAULT)
+ use_large_llq &= (llq->entry_size_recommended ==
+ ENA_ADMIN_LIST_ENTRY_SIZE_256B);

I'm not really a fan of using &= on bools, especially in multiple stages like this. I would write this with a switch (ena_force_large_llq_header) followed by if (!(...supported & ...SIZE_256B)) use_large_llq = false; since that seems to capture the intent more clearly?

(Also, I *assume* the device will never recommend 256B while not supporting it, but putting the "supported" test last is probably safer and certainly
clearer.)

If there's a strong preference within Amazon for the code style you used, I'm ok with it (FreeBSD is big on "play nice with upstreams"!) but if there's no particular reason for doing it this way then I think it could be improved.

Patch [1] ...
[1] - See Fixes
Fixes: 0a33c047a443 ("ena: Support LLQ entry size recommendation from
device")

I know every problem in computer science can be solved by adding more layers of indirection, but this seems unnecessary. How about just starting the commit message with "Commit 0a33c047a443 introduced new values to hw.ena.force_large_llq_header." ? (Definitely keep the Fixes: trailer though.)

Colin Percival

Added default: clause to switch to handle case of illegal values set to force_large_llq_header

This revision is now accepted and ready to land.Tue, Apr 29, 1:29 PM