Page MenuHomeFreeBSD

ZFSUmbrella
ActivePublic

Details

Description

Various work on OpenZFS and ZFS/FreeBSD.

Recent Activity

Jun 7 2019

avg closed D19094: Restore ARC MFU/MRU pressure.
Jun 7 2019, 6:36 AM · ZFS
avg accepted D19094: Restore ARC MFU/MRU pressure.
Jun 7 2019, 6:08 AM · ZFS

Jun 6 2019

slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#443654, @avg wrote:

Sorry, I myself went missing for a long while.
Yes, I can commit this change.
Do you want anything specific to appear in a commit message?
Like any additional attributions, etc?

Jun 6 2019, 7:17 PM · ZFS
avg added a comment to D19094: Restore ARC MFU/MRU pressure.

Sorry, I myself went missing for a long while.
Yes, I can commit this change.

Jun 6 2019, 11:08 AM · ZFS

May 11 2019

oshogbo added a watcher for ZFS: oshogbo.
May 11 2019, 1:33 PM

Apr 9 2019

ae added a reviewer for D19094: Restore ARC MFU/MRU pressure: ZFS.
Apr 9 2019, 4:16 PM · ZFS

Mar 11 2019

slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#418092, @avg wrote:

No more replays?

Unfortunately, no.
I think that we can commit your proposed change. If George comes up with a different and better solution later on, there should be no problem switching to it.

Mar 11 2019, 10:28 AM · ZFS
avg added a comment to D19094: Restore ARC MFU/MRU pressure.

No more replays?

Mar 11 2019, 9:39 AM · ZFS

Mar 7 2019

slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#412736, @avg wrote:

Do you succesefull contact George?

I've just got a reply from George.
He agrees with your analysis, but needs some more time to think about how to address the issue.
Let's wait a bit more.
Thanks!

Mar 7 2019, 10:08 AM · ZFS

Feb 22 2019

sef added inline comments to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
Feb 22 2019, 8:15 PM · ZFS

Feb 21 2019

avg added a comment to D19094: Restore ARC MFU/MRU pressure.

Do you succesefull contact George?

Feb 21 2019, 5:36 PM · ZFS

Feb 19 2019

slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#410508, @avg wrote:

Let me try to contact George again.

Feb 19 2019, 9:46 PM · ZFS

Feb 14 2019

avg added a comment to D19094: Restore ARC MFU/MRU pressure.

Let me try to contact George again.

Feb 14 2019, 12:27 PM · ZFS

Feb 13 2019

slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#410239, @mav wrote:

Don't sure about calling remove_reference() from arc_hdr_alloc_pabd() (or from parallel tasks), but see at ARC MFU/MRU size calculation in arc_change_state() called from arc_access() and !GHOST_STATE(state) case in arc_get_data_impl() called from arc_hdr_alloc_pabd().
I mean interchange this lines can cause problems for this accountings.

I am not sure what accounting problem you are talking about,

Feb 13 2019, 7:08 PM · ZFS
mav added a comment to D19094: Restore ARC MFU/MRU pressure.

Don't sure about calling remove_reference() from arc_hdr_alloc_pabd() (or from parallel tasks), but see at ARC MFU/MRU size calculation in arc_change_state() called from arc_access() and !GHOST_STATE(state) case in arc_get_data_impl() called from arc_hdr_alloc_pabd().
I mean interchange this lines can cause problems for this accountings.

Feb 13 2019, 5:53 PM · ZFS
slw_zxy.spb.ru added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#410062, @mav wrote:

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer.

Feb 13 2019, 2:15 PM · ZFS

Feb 12 2019

mav added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#410076, @lev wrote:

Huh? As far as I can see, problem not remove_reference(), but arc_adapt() called too late, with header in wrong state (promoted from something-ghost to live LRU), which brraks main idea of ARC adaptation.

Feb 12 2019, 11:01 PM · ZFS
lev added a comment to D19094: Restore ARC MFU/MRU pressure.
In D19094#410062, @mav wrote:

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer. I agree that according to logic of remove_reference() dropping last reference for header in ghost state is a failure, but how can remove_reference() be called before the arc_access() just on following line? I would guess from description telling about the case of prefetch read it should happen no sooner then we actually initiate the I/O, which is done much later then those two lines. So while I agree it is somewhat odd to have buffer for header in ghost state, is that a criminal.

Feb 12 2019, 10:36 PM · ZFS
mav added a comment to D19094: Restore ARC MFU/MRU pressure.

While I see the problem you are fixing, the fix looks ugly to me, that is why I would look for something nicer. I agree that according to logic of remove_reference() dropping last reference for header in ghost state is a failure, but how can remove_reference() be called before the arc_access() just on following line? I would guess from description telling about the case of prefetch read it should happen no sooner then we actually initiate the I/O, which is done much later then those two lines. So while I agree it is somewhat odd to have buffer for header in ghost state, is that a criminal.

Feb 12 2019, 9:54 PM · ZFS

Feb 6 2019

slw_zxy.spb.ru created D19094: Restore ARC MFU/MRU pressure.
Feb 6 2019, 2:48 PM · ZFS

Jan 24 2019

rgrimes added a comment to D18541: Add ZFS usage examples to the freebsd-tips file.

Thank you for the additions! Just my $0.05 worth of nit picks post commit.

Jan 24 2019, 8:04 PM · ZFS
bcr closed D18541: Add ZFS usage examples to the freebsd-tips file.
Jan 24 2019, 6:13 PM · ZFS

Jan 17 2019

bcr updated the diff for D18541: Add ZFS usage examples to the freebsd-tips file.

Update the diff based on Allan's comments. I've decided to remove the newsyslog compression example. This might be something better suited for the handbook. In it's place, I've added Allan's zfs snapshot range deletion examples. I've also changed the date in the custom property example to a more prominent date. ;-)

Jan 17 2019, 8:16 PM · ZFS

Jan 15 2019

bcr added a comment to D18541: Add ZFS usage examples to the freebsd-tips file.

Thanks for your feedback, @allanjude. I'll work on an updated patch that incorporates it.

Jan 15 2019, 4:35 PM · ZFS
allanjude accepted D18541: Add ZFS usage examples to the freebsd-tips file.
Jan 15 2019, 4:06 PM · ZFS

Dec 23 2018

bcr added reviewers for D18541: Add ZFS usage examples to the freebsd-tips file: 0mp, avg.

Adding more reviewers for more pairs of eyes for eventual approval.

Dec 23 2018, 6:52 PM · ZFS

Dec 14 2018

bcr updated the diff for D18541: Add ZFS usage examples to the freebsd-tips file.

Update with suggestions by Dru.

Dec 14 2018, 7:17 PM · ZFS
dru added a comment to D18541: Add ZFS usage examples to the freebsd-tips file.

These look great!

Dec 14 2018, 7:14 PM · ZFS

Dec 13 2018

bcr created D18541: Add ZFS usage examples to the freebsd-tips file.
Dec 13 2018, 6:20 PM · ZFS

Nov 9 2018

mav added a member for ZFS: mav.
Nov 9 2018, 5:59 PM

Oct 12 2018

mav closed D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 12 2018, 3:14 PM · ZFS
allanjude accepted D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 12 2018, 2:00 AM · ZFS
allanjude added inline comments to D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 12 2018, 1:48 AM · ZFS

Oct 11 2018

mav added inline comments to D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 11 2018, 9:16 PM · ZFS
sef added inline comments to D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 11 2018, 9:13 PM · ZFS
mav created D17523: Make ZFS indirect (removed) vdevs support TRIM.
Oct 11 2018, 9:06 PM · ZFS

Aug 29 2018

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  1. ARC don't rised, memory pressure does not arise, page daemon not activated.

The ARC is not growing after 8, but the ARC hit rate is too low. Why is it not growing? Is it because the free_memory < (arc_c >> arc_no_grow_shift) condition is true, or is there some other reason?

Aug 29 2018, 7:57 PM · ZFS
markj added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#361135, @markj wrote:

No, lots of memory in mbuf cache independ of memory pressure/page daemon in my workload:

  1. net workload raise
  2. mbuf consumption rise
  3. free memory reduction
  4. ARC reacts to free memory reduction by ARC reduction
  5. net workload reduction started
  6. mbuf released to zone cache
  7. free memory don't raised
  8. ARC don't rised, memory pressure does not arise, page daemon not activated.
Aug 29 2018, 7:37 PM · ZFS

Aug 28 2018

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#361135, @markj wrote:

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

No.
arc_available_memory(EXCLUDE_ZONE_CACHE) check conditions for memory pressure, check how many free memory see by OS (and kmem cache not counted for this).

Yes, which is exactly what the computation freemem - zfs_arc_free_target is. If you expand these definitions, it is vm_cnt.v_free_count - vm_cnt.v_free_target, where v_free_count does not include UMA caches. When v_free_count < v_free_target, the system is under memory pressure, and the page daemon attempts to free pages until v_free_count >= v_free_target. In -CURRENT, you can think of needfree as being the same as v_free_target - v_free_target when this difference is positive. In stable branches this is not quite true.

Aug 28 2018, 11:12 PM · ZFS
markj added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

No.
arc_available_memory(EXCLUDE_ZONE_CACHE) check conditions for memory pressure, check how many free memory see by OS (and kmem cache not counted for this).

Aug 28 2018, 9:04 PM · ZFS

Aug 23 2018

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

To be clear, I'm just stating that r332365 changed zfs_arc_free_target to be equal vm_cnt.v_free_target. It looks to me that this is equivalent to the change you made to arc_available_memory(EXCLUDE_ZONE_CACHE), where v_free_target is referenced directly.

Aug 23 2018, 10:04 PM · ZFS
markj added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#358922, @markj wrote:
  • The FMR_NEEDFREE target in -CURRENT is now the same as in this patch (though I didn't realize that when I made the change). See r332365.

Are you sure? A im still see 'r = FMR_LOTSFREE;' at line 4624
r332365: Discussed. Not very impotant. I am don't touch this is patch, may be zfs_arc_free_target is badly nameed (this target used as zfs_arc_min_free_target and I am use for my setup v_free_targetx1.4).
Any way, this is sysctl and can be tuned.

Aug 23 2018, 3:12 PM · ZFS
slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#358922, @markj wrote:

Sorry that this review has stalled lately. I would like to compare this patch to what's in -CURRENT, which has evolved a fair bit since the patch was updated. Once that picture is more clear, we can focus on stable/11.

Aug 23 2018, 1:26 PM · ZFS

Aug 22 2018

markj added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Sorry that this review has stalled lately. I would like to compare this patch to what's in -CURRENT, which has evolved a fair bit since the patch was updated. Once that picture is more clear, we can focus on stable/11.

Aug 22 2018, 10:33 PM · ZFS

Aug 7 2018

mmacy added a reviewer for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD: jeff.
Aug 7 2018, 9:26 PM · ZFS
rdeiriar_gmail.com added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I can confirm that the patch solves the issues as described in the "lightly loaded system eats swap space" thread on freebsd-stable mailing list.
On one of my desktop systems the issue is particularly noticeable as the disk subsystem is rather slow, the system became unusable after a
weekend sitting idle. I have not experienced the issues anymore after applying the patch

Aug 7 2018, 5:04 PM · ZFS

Jul 9 2018

peixotocassiano_gmail.com added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Hi there,

Jul 9 2018, 1:41 PM · ZFS

Jul 7 2018

me_cschwarz.com added a comment to D15748: Support destroy of multiple bookmarks on a single dataset..

What else is preventing this from getting merged?

Jul 7 2018, 9:02 PM · ZFS

Jun 30 2018

allanjude added a reviewer for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD: sef.
Jun 30 2018, 3:41 PM · ZFS

Jun 18 2018

FreeBSD_ShaneWare.Biz added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I am looking at your comment for PR163461. 70-80% of wired memory don't have direct relation to system slowdown or hard reset (for example "Mem: 3107M Active, 10G Inact, 231G Wired, 42K Buf, 5802M Free" and no slowdown), thes symptomps are caused by very low free_memory.
In my patch I try to convert unused wired ARC memory to system-wide free memory early.

Jun 18 2018, 10:21 PM · ZFS