Page MenuHomeFreeBSD

ice: Quiesce VFs before device reset
Needs ReviewPublic

Authored by kbowling on Thu, Aug 20, 1:47 AM.
Tags
None
Referenced Files
F167825801: D59024.id184427.diff
Mon, Aug 24, 7:41 PM
F167806108: D59024.diff
Mon, Aug 24, 4:50 PM
Unknown Object (File)
Sun, Aug 23, 4:23 PM
Unknown Object (File)
Sat, Aug 22, 10:36 PM
Unknown Object (File)
Sat, Aug 22, 9:46 AM
Unknown Object (File)
Fri, Aug 21, 4:07 PM
Unknown Object (File)
Fri, Aug 21, 4:06 PM
Unknown Object (File)
Fri, Aug 21, 12:02 PM
Subscribers

Details

Reviewers
kgalazka
Group Reviewers
Intel Networking
Restricted Owners Package(Owns No Changed Paths)
Summary
Reset preparation notifies cooperative VFs, then immediately releases
queue maps and firmware topology.  A VF which ignores the notification
can continue DMA while the PF tears down the resources which describe
it.

Assert VFSWR for each configured VF before teardown.  Run the mandatory
firmware drain serially, disable active receive queues, verify that PCIe
transactions have drained, and leave the VF held until its VSI rebuild
succeeds.  Block ordinary mailbox requests as soon as quiesce begins so
a hostile VF cannot re-enable queues in the warning interval.

Also clear VFLR status only after VFRD and perform the final Transaction
Pending check before publishing VFACTIVE.  This follows the VF reset
flow in section 4.1.3.3.3 of the Intel E810 Datasheet.  Serializing VFs
stays below the documented limit of four concurrent VM/VF reset flows.

Validated on an E810-XXV with active host VFs and a Linux passthrough
VF.  Two active VFs recovered together after a PF reset and eight after
a CORE reset; every VF returned initialized and traffic resumed.  A
128-VF configured passthrough topology also survived a PF reset.

With the bhyve process paused while ingress targeted an active Linux
VF, the PF completed quiesce, topology teardown, and VSI rebuild without
guest mailbox cooperation.  The VF remained configured but uninitialized
until the guest resumed, then re-handshook and restored traffic.

The companion failure-injection commit forced each mandatory drain
stage to fail.  Every affected VF remained inactive with bus mastering
disabled and recovered after SR-IOV recreation.

MFC after:      2 weeks
Sponsored by:   BBOX.io

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped