Page MenuHomeFreeBSD

mlx5/mlx5ib: Generating Queue Packets and Fixing Priority
Needs ReviewPublic

Authored by vag.singh_oneconvergence.com on Sep 27 2021, 12:15 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Apr 16, 3:47 PM
Unknown Object (File)
Dec 20 2023, 2:36 AM
Unknown Object (File)
Oct 30 2023, 10:46 PM
Unknown Object (File)
Jul 15 2023, 10:35 PM
Unknown Object (File)
Jun 26 2023, 11:33 PM
Unknown Object (File)
Jun 12 2023, 1:50 PM
Unknown Object (File)
May 20 2023, 1:07 AM
Unknown Object (File)
May 19 2023, 10:23 PM

Details

Reviewers
kib
hselasky
Summary

[PATCH 01/31] FreeBSD OFED support for DPDK MLX5 PMD

a) Within the mlx5_ib driver, the kernel erroneously used the wrong
function when asking the firmware to generate a queue packet

  • In particular, it requested MLX5_RX_HASH_FUNC_TOEPLITZ (= 0x1) rather than its Linux equivalent MLX5_RX_HASH_FN_TOEPLITZ (= 0x2) used, causing the error to show since the firmware expected 0x2 to perform the Toeplitz function.
  • The driver was supposed to use the preexisting variable MLX5_TIRC_RX_HASH_FN_HASH_TOEPLITZ (= 0x2), so changes were made to fix the generation of queue packets.

b) When the mlx5_core driver loads, it generates a set of RX flow tables
in a tree/hierarchical format for organizing namespaces.

  • A standard namespace called BYPASS is used within the kernel driver to insert flow rules into virtual port RX flow tables
  • On Linux, this namespace is populated with 16 levels, 8 occupied by "regular" priorities and 8 occupied by "don’t trap rules" priorities.
  • While by default on Linux the namespace gets automatically populated by these priorities, FreeBSD manually adds the regular priorities 1 by 1 into the tree, but fails to add the Don’t Trap priorities.
  • Consequently, FreeBSD kernel drivers crash when MLX5-PMD requests a regular priority level 7, as internal conversions cause the number to overflow beyond the max supported priorities.
  • Solution: Since FreeBSD manually adds each priority level rather than the automated process on linux, added an additional 8 priorities to include support for Don’t Trap priorities and increased the tree size to hold the additional priorities.
Test Plan

Tested on KVM, ESXi, and HyperV/Azure Hypervisors with FreeBSD 14.0-CURRENT and FreeBSD12.1-STABLE.

  • iperf
  • DPDK MLX5 PMD/testpmd

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c
73

After this change, MLX5_NUM_BYPASS_FTS is no longer used anywhere, from what I can see, and should be removed from:

sys/dev/mlx5/device.h:#define MLX5_NUM_BYPASS_FTS	9