Page MenuHomeFreeBSD

MFZol: Fix performance of "zfs recv" with many deletions
ClosedPublic

Authored by asomers on Oct 4 2019, 3:20 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Mar 29, 3:12 PM
Unknown Object (File)
Jan 2 2024, 3:07 PM
Unknown Object (File)
Dec 20 2023, 2:01 AM
Unknown Object (File)
Dec 2 2023, 7:00 PM
Unknown Object (File)
Nov 9 2023, 9:32 AM
Unknown Object (File)
Nov 7 2023, 9:09 PM
Unknown Object (File)
Sep 16 2023, 6:32 PM
Unknown Object (File)
Jul 22 2023, 9:10 AM

Details

Summary

MFZol: Fix performance of "zfs recv" with many deletions

This patch fixes 2 issues with the DMU free throttle implemented
in dmu_free_long_range(). The first issue is that get_next_chunk()
was calculating the number of L1 blocks the free would dirty
incorrectly. In some cases involving extremely large files, this
code would greatly overestimate the number of effected L1 blocks,
causing excessive calls to txg_wait_open(). This patch corrects
the calculation.

The second issue is that the free throttle uses the total number
of free'd blocks in all (open, quiescing, and syncing) txgs to
determine whether to throttle. This causes large frees (such as
those created by the first issue) to cause 4 txg syncs before
any further frees were allowed to proceed. This patch ensures
that the accounting is done entirely in a per-txg fashion, so
that frees from a given txg don't affect those that immediately
follow it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
zfsonlinux/zfs@f4c594da94d856c422512a54e48070f890b2685b

Freeing throttle should account for holes

Deletion throttle currently does not account for holes in a file.
This means that it can activate when it shouldn't.
To fix it we switch the throttle to be based on the number of
L1 blocks we will have to dirty when freeing

Reviewed-by: Tom Caputi <tcaputi@datto.com>
Reviewed-by: Matt Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alek Pinchuk <apinchuk@datto.com>
zfsonlinux/zfs@65282ee9e06b130f1f0169baf5d9bf0dd8fc1ef9

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 26865
Build 25184: arc lint + arc unit

Event Timeline

Update tunable description, too.

This revision is now accepted and ready to land.Oct 4 2019, 5:00 PM

This patch improves performance by fixing bugs in the freeing throttle. It's a combination of two patches that have been in ZoL since May (ZoL v0.8.0).

The way the freeing throttle is implemented now is susceptible to the overestimation of the data to be free which means artificially slow freeing of data.

One place where this slow freeing is a noticeable problem is a zfs recv when there are lots of FREE records present in the stream as it can make the recv speed very, very slow.