Page MenuHomeFreeBSD

ZFS sorted scans
ClosedPublic

Authored by sef on May 24 2018, 8:00 PM.

Details

Reviewers
mav
Group Reviewers
ZFS
Commits
rS334844: This originated from ZFS On Linux, as
Summary

This originated from ZFS On Linux, as https://github.com/zfsonlinux/zfs/commit/d4a72f23863382bdf6d0ae33196f5b5decbc48fd

During scans (scrubs or resilvers), it sorts the blocks in each transaction group by block offset; the result can be a significant improvement. (On my test system just now, which I put some effort to introduce fragmentation into the pool since I set it up yesterday, a scrub went from 1h2m to 33.5m with the changes.) I've seen similar rations on production systems.

FreeNAS has had these changes since Oct 2017.

Test Plan

Scrub & rebuild pools. Note times for performance analysis.

The pools are compatible with systems without the changes, so bouncing back and forth between two versions is possible, and I've used that for correctness-checking.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

sef created this revision.May 24 2018, 8:00 PM
avg added a subscriber: avg.May 25 2018, 6:31 AM

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

sef added a comment.May 25 2018, 6:33 AM
In D15562#328648, @avg wrote:

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

Same one, I believe. (Note that I started with the ZoL patches the middle of last year.)

sef updated this revision to Diff 43032.May 26 2018, 2:42 AM

A late sequential-scan-related patch was mentioned when I created the PR for OpenZFS, it's at https://github.com/zfsonlinux/zfs/commit/a8b2e30685c9214c

mav added a comment.May 26 2018, 3:03 AM

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

sef added a comment.May 26 2018, 3:06 AM
In D15562#329076, @mav wrote:

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

I thought about that, but I figured since I'm already getting this change in, and I added it to my PR for OpenZFS, I may as well do it here as well.

I can remove it easily enough.

sef updated this revision to Diff 43455.Jun 8 2018, 2:18 PM

Alexander pointed out that what I'd undone a change (range_tree_create()) because I'd ported from our 11-base system. Fixed that, and scrubbed a system a few times.

sef added a comment.Jun 8 2018, 5:39 PM

Checked in as r334844.

mav accepted this revision.Jun 8 2018, 7:58 PM
This revision is now accepted and ready to land.Jun 8 2018, 7:58 PM
This revision was automatically updated to reflect the committed changes.