HomeFreeBSD

Optimize RAIDZ expansion

Description

Optimize RAIDZ expansion

  • Instead of copying one ashift-sized block per ZIO, copy as much

as we have contiguous data up to 16MB per old vdev. To avoid data
moves use gang ABDs, so that read ZIOs can directly fill buffers
for write ZIOs. ABDs have much smaller overhead than ZIOs in both
memory usage and processing time, plus big I/Os do not depend on
I/O aggregation and scheduling to reach decent performance on HDDs.

  • Reduce raidz_expand_max_copy_bytes to 16MB on 32bit platforms.
  • Use 32bit range tree when possible (practically always now) to

slightly reduce memory usage.

  • Use ZIO_PRIORITY_REMOVAL for early stages of expansion, same as

for main ones.

  • Fix rate overflows in zpool status reporting.

With these changes expanding RAIDZ1 from 4 to 5 children I am able
to reach 6-12GB/s rate on SSDs and ~500MB/s on HDDs, both are
limited by devices instead of CPU.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #15680
Closes #16819

Details

Provenance
mavAuthored on Dec 6 2024, 4:50 PM
GitHub <noreply@github.com>Committed on Dec 6 2024, 4:50 PM
Parents
rGe8b333e4d33b: Fix false assertion in dmu_tx_dirty_buf() on cloning
Branches
Unknown
Tags
Unknown