mlx5e_get_wqe_sz(): return calculated max packet size
mlx5: define the strided RQ stride size
mlx5en: split mlx5e_build_rx_mbuf() into two parts
Also set the packet length before the HW LRO header update. For IPv6,
the update calls in6_cksum_partial_l2(), which asserts that the packet
length covers the TCP segment, while a small packet copied into a new
mbuf still has zero length at that point.
mlx5en: take the HW LRO IP length from the packet header length
mlx5e_lro_update_hdr() computed the IP total length from the raw CQE
byte_cnt. That is the frame length only for the cyclic RQ. For the
strided RQ, byte_cnt also carries the consumed strides count and the
filler flag in the upper bits, and the octet count includes the two
bytes of the two_byte_shift_en padding.
The caller now sets m_pkthdr.len to the frame length before the
header update, so use it instead. There is no functional change
for the existing RQ.
mlx5en: extract code to materialize received packet into mbuf
mlx5en: refill the IPsec rx tag of the receive buffer posted again
With the IPsec offload, mlx5e_alloc_rx_wqe() caches a preallocated tag
in each receive slot, and mlx5e_accel_ipsec_handle_rx() moves it to the
mbuf of a received IPsec packet. The cache is refilled when the buffer
of the slot is replaced, which happens when the buffer is passed up with
the packet. A buffer which is not passed up is posted again as is.
An IPsec packet is received in another mbuf, and that mbuf takes the
tag cached in the slot, when the pfil hook returns PFIL_REALLOCED for
the packet. The buffer stays in the ring and nothing refills the tag.
The next IPsec packet received into the buffer is passed up untagged,
with the warning from mlx5e_accel_ipsec_handle_rx_cqe(), although the
HW has already decrypted it. While the filter keeps reallocating the
packets, the buffer is never replaced, and no later IPsec packet
received into it gets the tag.
Refill the tag of the slot when its buffer is posted again. The check
of mlx5_accel_ipsec_rx_tag_add() is open-coded, to avoid the function
call for each buffer posted again when the IPsec offload is not used.
Fixes: d00f3505efad ("mlx5en: do now waste ipsec_accel_in_tag on non-ipsec packets rx")
mlx5en: free the cached IPsec rx tag regardless of the slot mbuf
The IPsec tag is cached in the receive slot. It is allocated when the
slot is posted, and stays cached until an IPsec packet consumes it. The
mbuf of the slot can be passed up by a packet which does not consume the
tag, and then stay absent if the allocation of the replacement fails.
As a result, mlx5e_destroy_rq() leaked the tag of a slot which had no
mbuf at the time of the destruction.
Free the cached tag independently of the slot mbuf.
Fixes: d00f3505efad ("mlx5en: do now waste ipsec_accel_in_tag on non-ipsec packets rx")
mlx5: define the linked-list striding RQ type and CQE byte count fields
mlx5en: implement the strided RQ receive
With the striding RQ (dev.mce.N.conf.striding_rq), the receive buffers
are wired pages allocated by the driver directly from VM instead of
the jumbo mbuf clusters. Each WQE is a buffer of 512 strides of 128
bytes, built from pages, and hosts many packets, including the HW LRO
aggregates. The received packet is wrapped into a chain of external
mbufs, each referencing the strides of the packet in a page, and the
page is released when the last mbuf using it is freed. Small packets are
copied.
The WQE is returned to the SW ownership when all its strides are
consumed, as tracked by the byte count of the completion entries,
including the error completions. The RX CQ is sized to accommodate
the completion per stride.
The feature requires the direct map, and it is mutually exclusive with
CQE zipping. The IPsec offload is not supported for the strided RQ,
the packets are dropped.
mlx5en: exclude the two-byte shift from the strided RQ packet length
With two_byte_shift_en, the hardware writes two padding bytes before
each packet in the stride, and the CQE octet count includes them.
The packet length was therefore two bytes too long. For a normal
packet, ip_input() trims the trailing garbage. For a HW LRO
aggregate, mlx5e_lro_update_hdr() extends the IP length over the two
extra bytes, and the TCP checksum offload flags make the stack accept
them as payload. The receiver then acknowledges two bytes that the
sender never sent, and the connection either silently loses data or
enters an endless ACK exchange with the sender.
Subtract the padding from the octet count.
mlx5en: support the IPsec offload receive with the strided RQ
The IPsec offload needs a tag for each received packet. The legacy RQ
preallocates the tag for the receive buffer, which is not possible for
the strided RQ, because its WQE is shared by many packets.
Allocate the tag when the CQE reports an IPsec flow, before the packet
is passed to pfil, copied, or wrapped, since all these paths end in
mlx5e_postproc_rx_mbuf(), which consumes the tag. If the tag cannot be
allocated, drop the packet and count it in the wqe_err statistic, as
the other drops of the strided RQ. The HW has already decrypted it, so
it cannot be passed up untagged. A tag which was not consumed stays
cached in the WQE slot and is freed by mlx5e_destroy_rq().
mlx5en: count the number of pages wired for strided rq rx mbufsDetails
Details
- Reviewers
slavash aehrenberg_nvidia.com gallatin
Diff Detail
Diff Detail
- Repository
- rG FreeBSD src repository
- Lint
Lint Skipped - Unit
Tests Skipped