ZFSUmbrella
ActivePublic

Details

Description

Various work on OpenZFS and ZFS/FreeBSD.

Recent Activity

Mon, Jun 18

FreeBSD_ShaneWare.Biz added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I am looking at your comment for PR163461. 70-80% of wired memory don't have direct relation to system slowdown or hard reset (for example "Mem: 3107M Active, 10G Inact, 231G Wired, 42K Buf, 5802M Free" and no slowdown), thes symptomps are caused by very low free_memory.
In my patch I try to convert unused wired ARC memory to system-wide free memory early.

Mon, Jun 18, 10:21 PM · ZFS
slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I have come to realise that there is another issue related to this, the default arc_max being wired ram that is not counted in max_wired means a default setup is allowed to wire more than the physical ram installed.

See my comment here for more explanation.

Mon, Jun 18, 1:53 PM · ZFS
FreeBSD_ShaneWare.Biz added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I have applied this patch to stable/11 r334820 and have been running without issue for 10 days.

Mon, Jun 18, 7:49 AM · ZFS

Sat, Jun 16

sef closed D15562: ZFS sorted scans.
Sat, Jun 16, 7:51 AM · ZFS

Thu, Jun 14

me_cschwarz.com added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

Checked all 'Done' boxes, hope this is the way to use Phabricator ¯\_(ツ)_/¯

Thu, Jun 14, 7:17 PM · ZFS
me_cschwarz.com updated the diff for D15748: Support destroy of multiple bookmarks on a single dataset..

Apply suggestions from review round 0.

Thu, Jun 14, 7:15 PM · ZFS

Tue, Jun 12

bcr added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

Thanks for the screenshot comparing the two man page renderings. I'm good with both ways.

Tue, Jun 12, 7:31 PM · ZFS
me_cschwarz.com added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

Addressed all complaints marked 'Done' in my private tree, will post a revision once @bcr responds to how I should handle the newline style vs. whitespace issue.

Tue, Jun 12, 7:19 PM · ZFS

Mon, Jun 11

mahrens requested changes to D15748: Support destroy of multiple bookmarks on a single dataset..
Mon, Jun 11, 5:49 PM · ZFS
bcr added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

Man page nit.

Mon, Jun 11, 5:27 PM · ZFS
allanjude added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

Adding range support for holds and bookmarks has been on my todo list for a while now.

Mon, Jun 11, 2:35 AM · ZFS

Sun, Jun 10

me_cschwarz.com retitled D15748: Support destroy of multiple bookmarks on a single dataset. from Summary: Support destroy of multiple bookmarks on a single dataset. to Support destroy of multiple bookmarks on a single dataset..
Sun, Jun 10, 9:27 PM · ZFS

Fri, Jun 8

mav accepted D15562: ZFS sorted scans.
Fri, Jun 8, 7:59 PM · ZFS
sef added a comment to D15562: ZFS sorted scans.

Checked in as r334844.

Fri, Jun 8, 5:39 PM · ZFS
sef updated the diff for D15562: ZFS sorted scans.

Alexander pointed out that what I'd undone a change (range_tree_create()) because I'd ported from our 11-base system. Fixed that, and scrubbed a system a few times.

Fri, Jun 8, 2:18 PM · ZFS

Sat, May 26

sef added a comment to D15562: ZFS sorted scans.
In D15562#329076, @mav wrote:

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

Sat, May 26, 3:06 AM · ZFS
mav added a comment to D15562: ZFS sorted scans.

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

Sat, May 26, 3:03 AM · ZFS
sef updated the diff for D15562: ZFS sorted scans.

A late sequential-scan-related patch was mentioned when I created the PR for OpenZFS, it's at https://github.com/zfsonlinux/zfs/commit/a8b2e30685c9214c

Sat, May 26, 2:42 AM · ZFS

Fri, May 25

sef added a comment to D15562: ZFS sorted scans.
In D15562#328648, @avg wrote:

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

Fri, May 25, 6:33 AM · ZFS
avg added a comment to D15562: ZFS sorted scans.

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

Fri, May 25, 6:31 AM · ZFS

Thu, May 24

sef created D15562: ZFS sorted scans.
Thu, May 24, 8:00 PM · ZFS

Tue, May 22

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Unified for -stable and current now

Tue, May 22, 3:49 PM · ZFS
lev added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

It is only solution to live-lock problem I encounter on my server when there are massive-parallel fast download.

Tue, May 22, 3:19 PM · ZFS

May 15 2018

mmacy added a watcher for ZFS: mmacy.
May 15 2018, 1:13 AM

May 11 2018

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Update to latest -STABLE changes

May 11 2018, 10:16 PM · ZFS

Apr 27 2018

kpraveen.lkml_gmail.com added a watcher for ZFS: kpraveen.lkml_gmail.com.
Apr 27 2018, 9:45 AM

Mar 14 2018

anthoine.bourgeois_blade-group.com added reviewers for D14669: Fix move handling of zvol devices out of txg sync thread: pjd, smh, ZFS.
Mar 14 2018, 10:31 AM · ZFS

Mar 12 2018

anthoine.bourgeois_blade-group.com updated the summary of D14669: Fix move handling of zvol devices out of txg sync thread.
Mar 12 2018, 7:24 PM · ZFS
anthoine.bourgeois_blade-group.com added a dependency for D14669: Fix move handling of zvol devices out of txg sync thread: D7179: move handling of zvol devices out of txg sync thread.
Mar 12 2018, 7:23 PM · ZFS
anthoine.bourgeois_blade-group.com created D14669: Fix move handling of zvol devices out of txg sync thread.
Mar 12 2018, 7:22 PM · ZFS

Jan 22 2018

seanc abandoned D12396: Default ABD chunk size.

@allanjude / @mahrens , it's worth pointing out that we eventually abandoned this change and went back to a 4K ABD chunk size. So while1K may have been more memory efficient in the short term, it ended up being suboptimal in the long run. I'm abandoning this issue and hoping no one repeats our lessons.

Jan 22 2018, 6:59 AM · ZFS
allanjude added a comment to D12396: Default ABD chunk size.

avg@ added the loader tunable to make this adjustable in https://svnweb.freebsd.org/base?view=revision&revision=323797 however I have every reason to believe (and measurements from production) that 4K is a going to result in more waste than a 1K default. A 1K default may result in fragmentation with 4K slabs but there is less waste for everyone with 1K than 4K.

@mahrens , do you think OpenZFS will go back to a 1K default?

Jan 22 2018, 3:40 AM · ZFS

Dec 19 2017

devnull_e-moe.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

just for whom it may concern: I've rolled back r326619 and reapplied slw patch on top - again, on "works for me" basics.

Dec 19 2017, 10:56 AM · ZFS

Dec 17 2017

junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Is there any intent to backport/MFC those changes into 11-STABLE?  That
I can probably test and adapt the code to more-quickly.

The needfree change is a good one, it's just that having it confined to
-HEAD causes problems for me right now due to "free computer of
appropriate configuration" constraints.

Thanks in advance.

I would expect the latest ZFS code to be merged from -current to stable/11 in time for 11.2-RELEASE

Dec 17 2017, 12:16 PM · ZFS

Dec 7 2017

thomsonk_yandex.com added a watcher for ZFS: thomsonk_yandex.com.
Dec 7 2017, 8:02 AM

Nov 19 2017

allanjude added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Is there any intent to backport/MFC those changes into 11-STABLE?  That
I can probably test and adapt the code to more-quickly.

The needfree change is a good one, it's just that having it confined to
-HEAD causes problems for me right now due to "free computer of
appropriate configuration" constraints.

Thanks in advance.

Nov 19 2017, 6:49 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Nov 19 2017, 6:43 PM · ZFS
allanjude added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Well that's a bitch.

It's easily fixed but I'm not going to be doing much with -HEAD until it
stabilizes and gets closer to being -RELEASE.

Nov 19 2017, 4:53 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Well that's a bitch.

Nov 19 2017, 2:18 PM · ZFS
junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Just a heads-up.
r325851 broke this (including Karl's updated patch at bug 187594).
"needfree" in arc.c has gone.

Nov 19 2017, 12:38 PM · ZFS

Oct 4 2017

devnull_e-moe.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

"works for me"
(survived make buildworld two times in a row ;-) kernel from clean sources hangs, as usual, somewhere in stage3/clang)

Oct 4 2017, 9:24 AM · ZFS

Oct 1 2017

junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Confirmed that the latest patch Karl uploaded on Bugzilla bug 187594 [1] (merged his patch with the one here) is...

Oct 1 2017, 12:14 AM · ZFS

Sep 30 2017

junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)

Sep 30 2017, 5:45 AM · ZFS

Sep 29 2017

karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)

Sep 29 2017, 6:28 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Definitely; I have the patch here in along with additional changes derived from my previous work running a soak test; should have some commentary and perhaps additional suggestions in a few days.

Sep 29 2017, 1:20 AM · ZFS

Sep 28 2017

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

As some of you probably know I've been chasing this same general issue here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

I'm playing with this patch set now on 11.1-STABLE (r324056) and other than leaving a crazy amount of inactive pages outstanding (which never get reclaimed in many instances, thus pressuring ARC size to half or so of what it could otherwise be) it appears to behave well.

I think I've got a fix for that last issue, and this patch set is a more-elegant approach to the UMA bloat problem then I had come up with. I want to run my changes to this code for a few days before contributing my thoughts in the form of code, but the short version is that adding a pager wakeup somewhat above the low-memory threshold appears to resolve the "frozen" inactive page issue and, if that proves up, this patch set looks very good and somewhat-superior to the one I have been running for ofa while (and thus a better option.)

Sep 28 2017, 10:30 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

As some of you probably know I've been chasing this same general issue here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

Sep 28 2017, 4:25 PM · ZFS

Sep 20 2017

mahrens added a comment to D12396: Default ABD chunk size.

@seanc OpenZFS/illumos has been at 4K chunk size since ABD was introduced. I'm not aware of any discussion around changing that.

Sep 20 2017, 3:55 PM · ZFS
seanc added a comment to D12396: Default ABD chunk size.

avg@ added the loader tunable to make this adjustable in https://svnweb.freebsd.org/base?view=revision&revision=323797 however I have every reason to believe (and measurements from production) that 4K is a going to result in more waste than a 1K default. A 1K default may result in fragmentation with 4K slabs but there is less waste for everyone with 1K than 4K.

Sep 20 2017, 2:42 PM · ZFS

Sep 19 2017

mav added a comment to D12396: Default ABD chunk size.

Fragmentation is a separate issue compared to the outright waste incurred with a compressed ARC and small-enough record sizes.

Sep 19 2017, 10:42 PM · ZFS