Page MenuHomeFreeBSD

net/tso: use the most restrictive segment size to perform the calculations
Needs RevisionPublic

Authored by royger on May 31 2016, 7:32 AM.

Details

Reviewers
gallatin
rrs
hselasky
Group Reviewers
network
transport
Summary

The current TSO segment size accounting code was not taking into account
that the link segment size can be smaller than the maximum segment size
supported by the network card. Without this change, the if_hw_tsomaxsegcount
limit could be ignored, and the mbuf chain created by the TSO code could
indeed have more segments than supported.

Sponsored by: Citrix Systems R&D

Test Plan

FreeBSD DomU PVHVM guests cannot 'route' traffic for other Xen PV guests on same Dom0 Host.
I am linking to the PR that this is suppose to be part of the fix for.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 4043
Build 4086: arc lint + arc unit

Event Timeline

royger retitled this revision from to net/tso: use the most restrictive segment size to perform the calculations.
royger updated this object.
royger edited the test plan for this revision. (Show Details)

I'm confused. t_maxsegsize is basically the connection's MSS, eg, what the NIC is supposed to break the TSO into. However if_hw_tsomaxsegsize is the max size of the TSO to be broken up. Eg, on a NIC without oddball limitations, if_hw_tsomaxsegcount might be 64, if_hw_tsomaxsegsize might be 65535. That means that TSO is supposed to send a max of 64K down to the NIC, in a max of 64 packets. If t_maxseg is less than 1000, then we'd have to limit the TSO to 64 packets.

It seems like by restricting if_hw_tsomaxsegsize to t_maxseg, you're essentially turning off TSO, since you're only allowing one MSS to be sent at a time.

I'm confused. t_maxsegsize is basically the connection's MSS, eg, what the NIC is supposed to break the TSO into. However if_hw_tsomaxsegsize is the max size of the TSO to be broken up. Eg, on a NIC without oddball limitations, if_hw_tsomaxsegcount might be 64, if_hw_tsomaxsegsize might be 65535. That means that TSO is supposed to send a max of 64K down to the NIC, in a max of 64 packets. If t_maxseg is less than 1000, then we'd have to limit the TSO to 64 packets.

It seems like by restricting if_hw_tsomaxsegsize to t_maxseg, you're essentially turning off TSO, since you're only allowing one MSS to be sent at a time.

Ugh, I'm wrong. I was confusing t_tsomax with t_tsomaxsegsize. I forgot that some NICs are apparently so broken that they limit TSO seg sizes.

I'm confused. t_maxsegsize is basically the connection's MSS, eg, what the NIC is supposed to break the TSO into. However if_hw_tsomaxsegsize is the max size of the TSO to be broken up. Eg, on a NIC without oddball limitations, if_hw_tsomaxsegcount might be 64, if_hw_tsomaxsegsize might be 65535. That means that TSO is supposed to send a max of 64K down to the NIC, in a max of 64 packets. If t_maxseg is less than 1000, then we'd have to limit the TSO to 64 packets.

It seems like by restricting if_hw_tsomaxsegsize to t_maxseg, you're essentially turning off TSO, since you're only allowing one MSS to be sent at a time.

Ugh, I'm wrong. I was confusing t_tsomax with t_tsomaxsegsize. I forgot that some NICs are apparently so broken that they limit TSO seg sizes.

So this is correct? Or am I missing something else?

I'm confused. t_maxsegsize is basically the connection's MSS, eg, what the NIC is supposed to break the TSO into. However if_hw_tsomaxsegsize is the max size of the TSO to be broken up. Eg, on a NIC without oddball limitations, if_hw_tsomaxsegcount might be 64, if_hw_tsomaxsegsize might be 65535. That means that TSO is supposed to send a max of 64K down to the NIC, in a max of 64 packets. If t_maxseg is less than 1000, then we'd have to limit the TSO to 64 packets.

It seems like by restricting if_hw_tsomaxsegsize to t_maxseg, you're essentially turning off TSO, since you're only allowing one MSS to be sent at a time.

Ugh, I'm wrong. I was confusing t_tsomax with t_tsomaxsegsize. I forgot that some NICs are apparently so broken that they limit TSO seg sizes.

So this is correct? Or am I missing something else?

I think it is correct, but I would prefer somebody from the transport team to review it.

Hi Drew,

t_tsomaxsegsize usually only affects the number of DMA segments and the mbuf chain. At least the Mellanox NICS do it this way.

How does t_maxseg affect the generated DMA chain?

--HPS

I mean DMA -> mbuf chain.

if_hw_tsomaxsegsize is used to calculate the size of the packets the NIC is going to emit, and tcp_m_copym uses it to solve other constraints (eg, if_hw_tsomaxsegcount)

I do think that this change is probably correct. If so, it needs to also be applied to rack and bbr.

(/rant) I honestly hate this weird impedance matching attempt between TSO and DMA. We should just do what linux does, and punt to software TSO for broken hardware that can't send something. (/end rant)

This revision now requires changes to proceed.Aug 25 2020, 10:55 PM