This is the error I get without the patch:

This patch fixes the build issue complaining about:

@neel_neelc.org can you tell me a bit more about your environment, make universe builds for all of us and only the powerpcspe tinderbox is currently broken because of these changes. I'm happy to commit these changes I just want to know what it's fixing.

In D19094#443654, @avg wrote:

Sorry, I myself went missing for a long while.
Yes, I can commit this change.

Do you want anything specific to appear in a commit message?
Like any additional attributions, etc?

Sorry, I myself went missing for a long while.
Yes, I can commit this change.

In D19094#418092, @avg wrote:

In D19094#417188, @slw_zxy.spb.ru wrote:

No more replays?

Unfortunately, no.
I think that we can commit your proposed change. If George comes up with a different and better solution later on, there should be no problem switching to it.

In D19094#417188, @slw_zxy.spb.ru wrote:

No more replays?

In D19094#412736, @avg wrote:

In D19094#411932, @slw_zxy.spb.ru wrote:

Do you succesefull contact George?

I've just got a reply from George.
He agrees with your analysis, but needs some more time to think about how to address the issue.
Let's wait a bit more.
Thanks!

In D19094#411932, @slw_zxy.spb.ru wrote:

Do you succesefull contact George?

In D19094#410508, @avg wrote:

Let me try to contact George again.

Let me try to contact George again.

In D19094#410239, @mav wrote:

In D19094#410192, @slw_zxy.spb.ru wrote:

Don't sure about calling remove_reference() from arc_hdr_alloc_pabd() (or from parallel tasks), but see at ARC MFU/MRU size calculation in arc_change_state() called from arc_access() and !GHOST_STATE(state) case in arc_get_data_impl() called from arc_hdr_alloc_pabd().
I mean interchange this lines can cause problems for this accountings.

I am not sure what accounting problem you are talking about,

In D19094#410192, @slw_zxy.spb.ru wrote:

Don't sure about calling remove_reference() from arc_hdr_alloc_pabd() (or from parallel tasks), but see at ARC MFU/MRU size calculation in arc_change_state() called from arc_access() and !GHOST_STATE(state) case in arc_get_data_impl() called from arc_hdr_alloc_pabd().
I mean interchange this lines can cause problems for this accountings.

In D19094#410062, @mav wrote:

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer.

In D19094#410076, @lev wrote:

Huh? As far as I can see, problem not remove_reference(), but arc_adapt() called too late, with header in wrong state (promoted from something-ghost to live LRU), which brraks main idea of ARC adaptation.

In D19094#410062, @mav wrote:

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer. I agree that according to logic of remove_reference() dropping last reference for header in ghost state is a failure, but how can remove_reference() be called before the arc_access() just on following line? I would guess from description telling about the case of prefetch read it should happen no sooner then we actually initiate the I/O, which is done much later then those two lines. So while I agree it is somewhat odd to have buffer for header in ghost state, is that a criminal.

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer. I agree that according to logic of remove_reference() dropping last reference for header in ghost state is a failure, but how can remove_reference() be called before the arc_access() just on following line? I would guess from description telling about the case of prefetch read it should happen no sooner then we actually initiate the I/O, which is done much later then those two lines. So while I agree it is somewhat odd to have buffer for header in ghost state, is that a criminal.

Thank you for the additions! Just my $0.05 worth of nit picks post commit.

Update the diff based on Allan's comments. I've decided to remove the newsyslog compression example. This might be something better suited for the handbook. In it's place, I've added Allan's zfs snapshot range deletion examples. I've also changed the date in the custom property example to a more prominent date. ;-)

Thanks for your feedback, @allanjude. I'll work on an updated patch that incorporates it.

Adding more reviewers for more pairs of eyes for eventual approval.

Update with suggestions by Dru.

These look great!

8. ARC don't rised, memory pressure does not arise, page daemon not activated.

The ARC is not growing after 8, but the ARC hit rate is too low. Why is it not growing? Is it because the free_memory < (arc_c >> arc_no_grow_shift) condition is true, or is there some other reason?

In D7538#361198, @slw_zxy.spb.ru wrote:

In D7538#361135, @markj wrote:

No, lots of memory in mbuf cache independ of memory pressure/page daemon in my workload:

1. net workload raise
2. mbuf consumption rise
3. free memory reduction
4. ARC reacts to free memory reduction by ARC reduction
5. net workload reduction started
6. mbuf released to zone cache
7. free memory don't raised
8. ARC don't rised, memory pressure does not arise, page daemon not activated.

In D7538#361135, @markj wrote:

In D7538#359546, @slw_zxy.spb.ru wrote:

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

No.

arc_available_memory(EXCLUDE_ZONE_CACHE) check conditions for memory pressure, check how many free memory see by OS (and kmem cache not counted for this).

Yes, which is exactly what the computation freemem - zfs_arc_free_target is. If you expand these definitions, it is vm_cnt.v_free_count - vm_cnt.v_free_target, where v_free_count does not include UMA caches. When v_free_count < v_free_target, the system is under memory pressure, and the page daemon attempts to free pages until v_free_count >= v_free_target. In -CURRENT, you can think of needfree as being the same as v_free_target - v_free_target when this difference is positive. In stable branches this is not quite true.

In D7538#359546, @slw_zxy.spb.ru wrote:

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

No.

arc_available_memory(EXCLUDE_ZONE_CACHE) check conditions for memory pressure, check how many free memory see by OS (and kmem cache not counted for this).

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

In D7538#359161, @slw_zxy.spb.ru wrote:

In D7538#358922, @markj wrote:

• The FMR_NEEDFREE target in -CURRENT is now the same as in this patch (though I didn't realize that when I made the change). See r332365.

Are you sure? A im still see 'r = FMR_LOTSFREE;' at line 4624

r332365: Discussed. Not very impotant. I am don't touch this is patch, may be zfs_arc_free_target is badly nameed (this target used as zfs_arc_min_free_target and I am use for my setup v_free_targetx1.4).
Any way, this is sysctl and can be tuned.

In D7538#358922, @markj wrote:

Sorry that this review has stalled lately. I would like to compare this patch to what's in -CURRENT, which has evolved a fair bit since the patch was updated. Once that picture is more clear, we can focus on stable/11.

Sorry that this review has stalled lately. I would like to compare this patch to what's in -CURRENT, which has evolved a fair bit since the patch was updated. Once that picture is more clear, we can focus on stable/11.

I can confirm that the patch solves the issues as described in the "lightly loaded system eats swap space" thread on freebsd-stable mailing list.
On one of my desktop systems the issue is particularly noticeable as the disk subsystem is rather slow, the system became unusable after a
weekend sitting idle. I have not experienced the issues anymore after applying the patch

Hi there,

What else is preventing this from getting merged?

In D7538#335854, @slw_zxy.spb.ru wrote:

I am looking at your comment for PR163461. 70-80% of wired memory don't have direct relation to system slowdown or hard reset (for example "Mem: 3107M Active, 10G Inact, 231G Wired, 42K Buf, 5802M Free" and no slowdown), thes symptomps are caused by very low free_memory.
In my patch I try to convert unused wired ARC memory to system-wide free memory early.

I have come to realise that there is another issue related to this, the default arc_max being wired ram that is not counted in max_wired means a default setup is allowed to wire more than the physical ram installed.

See my comment here for more explanation.

I have applied this patch to stable/11 r334820 and have been running without issue for 10 days.

Checked all 'Done' boxes, hope this is the way to use Phabricator ¯\_(ツ)_/¯

Apply suggestions from review round 0.

Thanks for the screenshot comparing the two man page renderings. I'm good with both ways.

Addressed all complaints marked 'Done' in my private tree, will post a revision once @bcr responds to how I should handle the newline style vs. whitespace issue.

Man page nit.

Adding range support for holds and bookmarks has been on my todo list for a while now.

Checked in as r334844.

Alexander pointed out that what I'd undone a change (range_tree_create()) because I'd ported from our 11-base system. Fixed that, and scrubbed a system a few times.

In D15562#329076, @mav wrote:

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

I don't think it is a very good idea to merge two pretty much independent ZoL commits into one FreeBSD commit. It could have sense if the first one would be significantly broken and second fixed the issue, but priority upgrade implemented in second commit is only a performance optimization, mostly unrelated to scrub even, in which case having it separate would make commit history cleaner for later comparison.

A late sequential-scan-related patch was mentioned when I created the PR for OpenZFS, it's at https://github.com/zfsonlinux/zfs/commit/a8b2e30685c9214c

In D15562#328648, @avg wrote:

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

Just a quick question, is this the same change that was recently presented at OpenZFS summit (http://open-zfs.org/w/images/a/a0/Saso_-_resilver_update.pdf)?
Or an alternative to it?

Unified for -stable and current now

It is only solution to live-lock problem I encounter on my server when there are massive-parallel fast download.

Update to latest -STABLE changes

@allanjude / @mahrens , it's worth pointing out that we eventually abandoned this change and went back to a 4K ABD chunk size. So while1K may have been more memory efficient in the short term, it ended up being suboptimal in the long run. I'm abandoning this issue and hoping no one repeats our lessons.

In D12396#257653, @seanc wrote:

avg@ added the loader tunable to make this adjustable in https://svnweb.freebsd.org/base?view=revision&revision=323797 however I have every reason to believe (and measurements from production) that 4K is a going to result in more waste than a 1K default. A 1K default may result in fragmentation with 4K slabs but there is less waste for everyone with 1K than 4K.

@mahrens , do you think OpenZFS will go back to a 1K default?

just for whom it may concern: I've rolled back r326619 and reapplied slw patch on top - again, on "works for me" basics.

In D7538#273971, @allanjude wrote:

In D7538#273970, @karl_denninger.net wrote:

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Is there any intent to backport/MFC those changes into 11-STABLE?  That
I can probably test and adapt the code to more-quickly.

The needfree change is a good one, it's just that having it confined to
-HEAD causes problems for me right now due to "free computer of
appropriate configuration" constraints.

Thanks in advance.

• smime.p7s4 KBDownload

I would expect the latest ZFS code to be merged from -current to stable/11 in time for 11.2-RELEASE

In D7538#273970, @karl_denninger.net wrote:

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Is there any intent to backport/MFC those changes into 11-STABLE?  That
I can probably test and adapt the code to more-quickly.

The needfree change is a good one, it's just that having it confined to
-HEAD causes problems for me right now due to "free computer of
appropriate configuration" constraints.

Thanks in advance.

• smime.p7s4 KBDownload

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

In D7538#273915, @karl_denninger.net wrote:

Well that's a bitch.

It's easily fixed but I'm not going to be doing much with -HEAD until it
stabilizes and gets closer to being -RELEASE.

• smime.p7s4 KBDownload

Well that's a bitch.

Just a heads-up.
r325851 broke this (including Karl's updated patch at bug 187594).
"needfree" in arc.c has gone.

"works for me"
(survived make buildworld two times in a row ;-) kernel from clean sources hangs, as usual, somewhere in stage3/clang)

Confirmed that the latest patch Karl uploaded on Bugzilla bug 187594 [1] (merged his patch with the one here) is...

In D7538#260193, @karl_denninger.net wrote:

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)