Thank you, I was looking at this too.

The whole watchdog thing, like mp_ring, is absurdly over-complicated for no real reason. Just do it the mxge way: if we have pending transmits (eg, producer != consumer) *AND* the nic has not consumed any packets since the last time the watchdog callout ran, then reset. I get dizzy every time i look at the iflib queue-state stuff. Its just extra complexity that adds nothing.

This fires too often, maybe some false positives. I will rework.

In D58266#1336569, @netchild wrote:

This fires too often, maybe some false positives. I will rework.

I've frankly lost track of how exactly the mp_ring path cleans things since I've been entirely focused on the simple_tx path. In that context, i can see how the mxge method would lead to false positives. It would be great if you could also test your changes with dev.$DRIVER.$UNIT.iflib.simple_tx=1. I plan to make that the default and hopefully remove the overly complicated mp_ring path entirely, I think removing mp_ring will make iflib much simpler and easier to maintain

Results from the affected hardware (em0, 82541PI), ~44 h
on a v3 kernel. Summary: 16 watchdog fires, all true positives, 0 false
positives.

@gallatin — I did try the plain mxge predicate first (pidx != cidx with
no hardware progress between watchdog cycles); that was the previous
version of this diff and it FAILED its field test. On the 8254x, WTHRESH
writeback coalescing withholds the final report-status writeback on a
quiet queue, so a perfectly healthy idle queue shows pidx != cidx with no
visible DD for seconds at a time. Result was a reset storm: fires every
~4.4 s through nighttime lulls.

The growth arming is what makes that verdict trustworthy. Arming requires
NEW work (unharvested writebacks grew), and new completions flush the
previously withheld writebacks, so by verdict time a healthy queue always
has visible DD. Measured: over a 13.2 h night (92169 samples) the kernel
reached the verdict ~100 times on healthy hardware and EVERY peek found
credits > 0 — never a reset. peek == 0 after growth means nothing came
back despite new work, i.e. a genuine hang.

On simple_tx specifically, your case is stronger than mine: I re-ran the
full six-regime drip matrix under dev.em.0.iflib.simple_tx=1 and the
predicate never armed at all — max_armed = 0 in every regime, vs. a peak
of 2 under mp_ring (threshold is 4). The inline reclaim in
iflib_simple_transmit advances ift_processed in the same period as any
growth, so the frozen condition cannot hold. IFLIB_WDOG_ARMED_PERIODS=4
is calibrated by the mp_ring worst case and needs no retune when
simple_tx becomes the default.

Related, and it argues for your removal plan: the armed kick
(GROUPTASK_ENQUEUE) is effectively a reclaim no-op under mp_ring — with
ift_db_pending == 0, _task_fn_tx only calls ifmp_ring_check_drainage(),
which returns immediately on an idle ring (mp_ring.c:475). No correctness
issue (the next transmit reclaims; the queue is ~2% occupied), but the
lost-interrupt recovery is best-effort there. Under simple_tx,
_task_fn_tx calls iflib_completed_tx_reclaim() directly
(iflib.c:4094-4098), so the recovery path becomes fully functional once
mp_ring goes away.

@markj — the "mark the queue idle on partial reclamation" concern no
longer applies: v3 does not track queue state at all. D58282 deletes
ift_qstatus and the IFLIB_QUEUE_* states outright. The predicate is now
just (link up && unharvested writebacks grew && stayed frozen for 4 timer
periods) -> peek the descriptors for credits. Nothing is marked idle, so
there is no partial-reclaim edge case to miss.

True-positive data, from a 48 h soak: night 1 clean (0 fires). Night 2 had
four spontaneous wedge episodes (8/3/1/4 fires); the watchdog self-healed
three of them in <=35 s each. The fourth was reinit-resistant — repeated
resets did not cure it and the box needed a reboot. I have seen that shape
twice now (a separate episode took 48 fires without curing), so I would
like input on whether this should carry a reset backoff or a cap rather
than retrying indefinitely; I did not want to bundle a policy decision
into this diff.

Negative control: after that reboot the box sat through 7+ h of
provider-side IPv4-only loss to the gateway (50-100% loss, v6 clean, em0
counters healthy) with watchdog_timeouts = 0. The predicate does not fire
on external loss.