l2arc_log_blkptr_valid() ends with (!evicted || dev->l2ad_first), which
disables the eviction-overlap test entirely on a first sweep. That test
is meaningless in that state, since l2arc_evict() returns immediately
and l2ad_evict never advances off l2ad_start, but dropping it leaves the
log block bounded only by device geometry. On a first sweep only the
region below the write hand has been written, so a block at or beyond
l2ad_hand describes data this incarnation never wrote.
l2arc_hdr_restore() performs no per-entry validation, so every such
entry inflates arcstat_l2_psize and, via vdev_space_update(), the cache
vdev's vs_alloc. Nothing reconciles it because l2arc_evict() never runs
on a first sweep, and once vs_alloc exceeds vs_space the unclamped
subtraction in zpool(8) wraps and the device reports 16.0E free. Stale
entries also let the L2ARC read offsets that were never written, which
then fail checksum verification.
Removing and re-adding a cache vdev is enough to set this up:
l2arc_add_vdev() resets l2ad_hand to l2ad_start while the previous
incarnation's log blocks remain higher up the device, and the backward
walk then crosses l2ad_start and wraps into them.
Bound the first-sweep case by the write hand instead of disabling the
check, and state the geometry conditions once rather than duplicating
them across both branches.
The 16.0E symptom has been reported since 2015. OpenZFS issue 10224
carries the only prior analysis, which suspected a leak in
l2arc_evict() and was closed incidentally by OpenZFS pull request 9789
rather than by a fix.
PR: 250323
External-issue: https://github.com/openzfs/zfs/issues/12779
External-issue: https://github.com/openzfs/zfs/issues/10224
External-issue: https://github.com/openzfs/zfs/issues/5583
External-issue: https://github.com/openzfs/zfs/issues/3400
External-issue: https://github.com/openzfs/zfs/issues/3114
Upstream fix: https://github.com/openzfs/zfs/pull/18827
Signed-off-by: Nick Price <nick@spun.io>