HomeFreeBSD

RAIDZ2/3 fails to heal silently corrupted parity w/2+ bad disks

Description

RAIDZ2/3 fails to heal silently corrupted parity w/2+ bad disks

When scrubbing, (non-sequential) resilvering, or correcting a checksum
error using RAIDZ parity, ZFS should heal any incorrect RAIDZ parity by
overwriting it. For example, if P disks are silently corrupted (P being
the number of failures tolerated; e.g. RAIDZ2 has P=2), zpool scrub
should detect and heal all the bad state on these disks, including
parity. This way if there is a subsequent failure we are fully
protected.

With RAIDZ2 or RAIDZ3, a block can have silent damage to a parity
sector, and also damage (silent or known) to a data sector. In this
case the parity should be healed but it is not.

The problem can be noticed by scrubbing the pool twice. Assuming there
was no damage concurrent with the scrubs, the first scrub should fix all
silent damage, and the second scrub should be "clean" (zpool status
should not report checksum errors on any disks). If the bug is
encountered, then the second scrub will repair the silently-damaged
parity that the first scrub failed to repair, and these checksum errors
will be reported after the second scrub. Since the first scrub repaired
all the damaged data, the bug can not be encountered during the second
scrub, so subsequent scrubs (more than two) are not necessary.

The root cause of the problem is some code that was inadvertently added
to raidz_parity_verify() by the DRAID changes. The incorrect code
causes the parity healing to be aborted if there is damaged data
(rc_error != 0) or the data disk is not present (!rc_tried). These
checks are not necessary, because we only call raidz_parity_verify()
if we have the correct data (which may have been reconstructed using
parity, and which was verified by the checksum).

This commit fixes the problem by removing the incorrect checks in
raidz_parity_verify().

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11489
Closes #11510

Details

Provenance
mahrensAuthored on Jan 27 2021, 12:05 AM
GitHub <noreply@github.com>Committed on Jan 27 2021, 12:05 AM
Parents
rGd7265b330954: ZTS: zpool_export test improvements
Branches
Unknown
Tags
Unknown