Page MenuHomeFreeBSD

ice: Rebuild VF VSIs after PF resets
Needs ReviewPublic

Authored by kbowling on Tue, Aug 18, 10:38 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Aug 19, 12:14 PM
Unknown Object (File)
Tue, Aug 18, 12:45 PM
Unknown Object (File)
Tue, Aug 18, 12:45 PM
Unknown Object (File)
Tue, Aug 18, 12:07 PM
Unknown Object (File)
Tue, Aug 18, 12:05 PM
Subscribers

Details

Reviewers
kgalazka
Group Reviewers
Intel Networking
Restricted Owners Package(Owns No Changed Paths)
Summary
PF and device resets discard all hardware VSI state.  ice_rebuild()
only recreated the main PF VSI before replaying configuration for every
VSI.  As a result, replay used stale VF VSI handles.  Firmware rejected
it with AQ_RC_EACCES and the failure aborted the entire PF rebuild.

Re-add each VF VSI before replaying its configuration, and do not
publish VFACTIVE until both operations succeed.  Clear the initialized
state so that the VF must negotiate again.  If a VF rebuild fails, leave
it inactive and continue so that one guest cannot prevent the PF or
sibling VFs from recovering.

Make that failed state authoritative in the mailbox path.  While the
firmware VSI is invalid, permit only VERSION and RESET_VF and reject
operations which require VSI state.  A VFR may complete the hardware
reset, but cannot make the PF-owned VSI valid or publish the VF active.

Preserve accumulated VF statistics while establishing a new raw
hardware sample after reconstruction.  This keeps the cumulative totals
returned to iavf monotonic across PF resets.  Return after rejecting a
GET_STATS request for the wrong VSI so that it cannot receive a second
success reply.

Before a locally initiated reset, notify initialized VFs with
RESET_IMPENDING while the mailbox control queue is still alive.  Ignore
individual send failures so one VF cannot prevent notification of its
siblings or the reset itself.  Send the event from the common reset
preparation path and before directly triggering CORE and GLOBAL resets.

Preserve the inactive state across later VF resets.  The zero-queue
Disable LAN Tx AQ remains mandatory to complete every VFR, but do not
clear VFSWR or publish VFACTIVE while the PF-owned VSI remains invalid.

Linux ice uses the same separation: generic rebuild excludes VF VSIs,
the VF reset path rebuilds them separately, and reset preparation
notifies initialized VFs before tearing down the control queues.

Validated on an E810-XXV with one and eight host-attached iavf VFs.
Repeated PF and CORE resets recovered every VF under traffic without
watchdog, MDD, or persistent data-path errors.  An additional one-VF
test kept traffic active across PF and CORE resets; carrier and traffic
recovered automatically and both PF and VF watchdog counters stayed at
zero.

Two four-queue VFs passed through to a Linux 7.0 iavf guest also
recovered carrier and traffic automatically after PF and CORE resets.
Simultaneous traffic on both VFs resumed without intervention and the PF
watchdog counter remained zero.

DPDK 25.11 testpmd, using vfio no-IOMMU and two queues per VF, sustained
traffic on each reset.  It received reset events for both
VFs after PF and CORE resets.  The documented ethdev stop, reset,
reconfigure, and start sequence restored traffic after each reset.
Physical bus mastering remained enabled and the PF watchdog counter
remained zero.

MFC after:      2 weeks

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

Owners added a reviewer: Restricted Owners Package.Tue, Aug 18, 10:38 AM
kbowling edited the summary of this revision. (Show Details)

Gates invalid/rebuild-failed VFs, preserve cumulative VF statistics across rebuilds, prevent double GET_STATS reply.

kbowling edited the summary of this revision. (Show Details)

add ice(4) documentation on VF reconstruction