Page MenuHomeFreeBSD

iavf: Handle queue disable during reset recovery
Needs ReviewPublic

Authored by kbowling on Tue, Aug 18, 10:32 AM.

Details

Reviewers
kgalazka
Group Reviewers
Intel Networking
Restricted Owners Package(Owns No Changed Paths)
Summary
Some PFs reject a redundant DISABLE_QUEUES request after queues have
already stopped.  The historical driver retried the request but
continued.  The mailbox recovery change instead treated this NACK as a
transport failure, tore down the Admin Queue without a VF reset, and
could not recover the interface during API rediscovery.

Track a PF's pre-reset warning separately from an observed reset and
from exclusive ownership of polled AdminQ replies.  RESET_IMPENDING
leaves asynchronous AdminQ processing available, so a concurrent stop
can consume its DISABLE_QUEUES reply.  VC_POLLING protects VERSION and
GET_VF_RESOURCES rediscovery without blocking that reply.

A reset warning alone does not prove queue DMA stopped.  Require either
a non-active RSTAT or a completed requested reset before entering reset
recovery.  If the AdminQ is unavailable after a fast reset whose
intermediate RSTAT state was missed, rebuild the virtchnl connection and
require an acknowledged queue disable or explicit VF reset.  Fall back
to fencing PCI DMA if neither operation can prove the queues stopped.

iflib releases queue DMA resources immediately after IFDI_STOP returns.
A PF error reply proves that virtchnl is alive, but not that its LAN
queues stopped.  Do not continue initialization or resource teardown
from an unconfirmed disable.  Recheck RSTAT, wait for a reset already in
progress, or explicitly request a VF reset and wait for completion.
If neither mechanism can stop the queues, clear bus mastering, wait for
pending PCIe transactions, and keep DMA fenced until driver reattach.

For a software-requested reset, publish VFR_STATE as in progress before
submitting RESET_VF, as required by E810 section 4.1.3.3.1.  This keeps
a stale VFACTIVE value from falsely completing the reset.  Propagate
submission failures, restore the prior state when no reset was
submitted, and delay before polling for completion.

Keep ordinary AdminQ processing out of observed reset recovery until a
replacement connection has been established.  Treat a verified PCI DMA
fence as terminal instead of scheduling mailbox retries which cannot
recover it.

Clear the attach-time recovery marker after the initial resource
handshake.  This prevents iflib's first stop/init cycle from discarding
a healthy mailbox.

Linux and DPDK likewise treat RESET_IMPENDING as a warning and skip
DISABLE_QUEUES during recognized reset recovery.  Their ordinary stop
paths treat disable failure as advisory (DPDK commit
3e6a5d2d310a56a86708afce29b99e00400c42d6), but iflib's immediate DMA
resource release requires stronger confirmation.

Validated on an E810-XXV with two four-queue host-attached VFs.  Fresh
attach, 20 stop/start cycles per VF, and recovery from PF and CORE
resets completed without mailbox rediscovery failures or watchdog
events.  With one VF carrying continuous traffic, PF and CORE resets
recovered carrier and traffic automatically without fencing DMA; both
watchdog counters remained zero.

MFC after:      2 weeks

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped