mlx5: fix deadlock in mlx5_fs_tree.c

[mm]

Summary

If removing a node of type FS_TYPE_FLOW_DEST we lock the flow group too late. This can lead to a deadlock and conflict with fs_add_dst_fg().
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274715

[gallatin]

Wouldn't it be better to toggle the IFCAP2_BIT(IFCAP2_RXTLS4) and IFCAP2_BIT(IFCAP2_RXTLS6) than to add the check in mlx5e_tls_rx_snd_tag_alloc()?

[mm]

Looks like we can already set "-rxtls" on network interfaces, it just isn't documented in ifconfig manual page.
So I am withdrawing this revision - the workaround is to disable rxtls.

[kib]

We (Nvidia) have the large rework of the flow steering code sitting in the internal branch. I hope to have the time to extract and prepare that code for commit later this week. Then, the fix could be no longer needed, or forward-ported to new fs code. I need to discuss it with the author.

sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c

1722

Then you would need to add an assert there?

[mm]

I have updated the patch with the assert.

[kib]

Linuxkpi mutexes are FreeBSD' sx.

[mm]

In other source files of the mlx5 driver the same "mtx_assert" syntax is used to assert sx locks, I wanted to stay consistent. If you want I can replace it with sx_assert().

[kib]

In D42368#967484, @mm wrote:

In other source files of the mlx5 driver the same "mtx_assert" syntax is used to assert sx locks, I wanted to stay consistent. If you want I can replace it with sx_assert().

Can you point out these places, please?

[mm]

Ok, so when I understand it correctly, in the linuxkpi the "struct mutex" is a shared/exclusive lock sx(9) and "struct mtx" is mutex for thread synchronization mutex(9).

I am updating the patch.

[mm]

I have tested the patch on a semi-production system with 30000 ifnet ktls sessions and 50Gbit/s traffic. Is it ok to go in?