ZFSUmbrella
ActivePublic

Details

Description

Various work on OpenZFS and ZFS/FreeBSD.

Recent Activity

Thu, Dec 7

thomsonk_yandex.com added a watcher for ZFS: thomsonk_yandex.com.
Thu, Dec 7, 8:02 AM

Sun, Nov 19

allanjude added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Is there any intent to backport/MFC those changes into 11-STABLE?  That
I can probably test and adapt the code to more-quickly.

The needfree change is a good one, it's just that having it confined to
-HEAD causes problems for me right now due to "free computer of
appropriate configuration" constraints.

Thanks in advance.

Sun, Nov 19, 6:49 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

I'll try to get some time allocated to that - right now the machine that
I use for this effort is busy doing other things but I may be able to
free up some time on it in the next week or two.

Sun, Nov 19, 6:43 PM · ZFS
allanjude added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Well that's a bitch.

It's easily fixed but I'm not going to be doing much with -HEAD until it
stabilizes and gets closer to being -RELEASE.

Sun, Nov 19, 4:53 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Well that's a bitch.

Sun, Nov 19, 2:18 PM · ZFS
junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Just a heads-up.
r325851 broke this (including Karl's updated patch at bug 187594).
"needfree" in arc.c has gone.

Sun, Nov 19, 12:38 PM · ZFS

Oct 4 2017

devnull_e-moe.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

"works for me"
(survived make buildworld two times in a row ;-) kernel from clean sources hangs, as usual, somewhere in stage3/clang)

Oct 4 2017, 9:24 AM · ZFS

Oct 1 2017

junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Confirmed that the latest patch Karl uploaded on Bugzilla bug 187594 [1] (merged his patch with the one here) is...

Oct 1 2017, 12:14 AM · ZFS

Sep 30 2017

junchoon_dec.sakura.ne.jp added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)

Sep 30 2017, 5:45 AM · ZFS

Sep 29 2017

karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Being new to phabricator.... how do I upload a diff that is off a (probably) different base rev and has my changes in it (I don't really want to "update" the existing diff, or do I?)

Sep 29 2017, 6:28 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Definitely; I have the patch here in along with additional changes derived from my previous work running a soak test; should have some commentary and perhaps additional suggestions in a few days.

Sep 29 2017, 1:20 AM · ZFS

Sep 28 2017

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

As some of you probably know I've been chasing this same general issue here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

I'm playing with this patch set now on 11.1-STABLE (r324056) and other than leaving a crazy amount of inactive pages outstanding (which never get reclaimed in many instances, thus pressuring ARC size to half or so of what it could otherwise be) it appears to behave well.

I think I've got a fix for that last issue, and this patch set is a more-elegant approach to the UMA bloat problem then I had come up with. I want to run my changes to this code for a few days before contributing my thoughts in the form of code, but the short version is that adding a pager wakeup somewhat above the low-memory threshold appears to resolve the "frozen" inactive page issue and, if that proves up, this patch set looks very good and somewhat-superior to the one I have been running for ofa while (and thus a better option.)

Sep 28 2017, 10:30 PM · ZFS
karl_denninger.net added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

As some of you probably know I've been chasing this same general issue here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

Sep 28 2017, 4:25 PM · ZFS

Sep 20 2017

mahrens added a comment to D12396: Default ABD chunk size.

@seanc OpenZFS/illumos has been at 4K chunk size since ABD was introduced. I'm not aware of any discussion around changing that.

Sep 20 2017, 3:55 PM · ZFS
seanc added a comment to D12396: Default ABD chunk size.

avg@ added the loader tunable to make this adjustable in https://svnweb.freebsd.org/base?view=revision&revision=323797 however I have every reason to believe (and measurements from production) that 4K is a going to result in more waste than a 1K default. A 1K default may result in fragmentation with 4K slabs but there is less waste for everyone with 1K than 4K.

Sep 20 2017, 2:42 PM · ZFS

Sep 19 2017

mav added a comment to D12396: Default ABD chunk size.

Fragmentation is a separate issue compared to the outright waste incurred with a compressed ARC and small-enough record sizes.

Sep 19 2017, 10:42 PM · ZFS

Sep 18 2017

seanc added a comment to D12396: Default ABD chunk size.

@mav I'm less concerned about fragmentation due to a changing workload. I'm much more concerned about the steady-state waste that ABD represents for smaller record sizes. Let me quote a recent issue where we investigated memory usage:

Sep 18 2017, 5:47 PM · ZFS
mav added a comment to D12396: Default ABD chunk size.

I'll second @mahrens worry about possible memory fragmentation. Plus FreeBSD kernel memory allocator does not even have the mentioned hooks to reallocate memory fighting it. Plus I still hope we can make GEOM vdev to use unmapped I/O to avoid extra buffer copying, which depends on blocks to be multiple of page size (4KB). I am not against making it loader tunable, but not ready to change the default or promote the tunable.

Sep 18 2017, 1:46 AM · ZFS

Sep 17 2017

mahrens added a comment to D12396: Default ABD chunk size.

FYI - we use 1K ABD chunks at Delphix (on illumos), and we've found that up to 40% of memory can end up being wasted because of memory fragmentation. At least on illumos, the 1K ABD kmem cache will use slab size 4K (one page), so there are 4 chunks per slab. After eviction, we may have many partially-full slabs, which wastes the free chunks. We are considering implementing a kmem_move callback which would allow the free chunks to be consolidated, reducing the potential for memory waste. But it isn't done yet.

Sep 17 2017, 2:58 AM · ZFS

Sep 16 2017

seanc created D12396: Default ABD chunk size.
Sep 16 2017, 11:40 PM · ZFS

May 15 2017

devnull_e-moe.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

somehow this patch solved my old problem - instant hangs somewhere in building libclang/libllvm while making buildworld.
my configuration: 11stable x64 on vmware vm , zfs (default layout by bsdinstall) 4 or 6G memory (no visible difference, no explicit prefetch on 4G setup), 4core parallel build
top usually reports something like "2G free" (until entire vm become nonresponsible)

May 15 2017, 4:03 PM · ZFS

Mar 18 2017

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  1. Restore lost code
  2. Conrtol (by sysctl) draining per-CPU UMA cache
  3. Skip first draining UMA cache for case of enough memory in caches.
  4. Don't wait drained memory accounted as free. Stop daining if expect as enough.
Mar 18 2017, 10:34 AM · ZFS

Mar 5 2017

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Fix userland compilation

Mar 5 2017, 1:43 PM · ZFS

Mar 4 2017

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Generalize zone travers algorithm and do some optimization on cache reclaim (arc_kmem_reap_now() costly and call only as last resort)

Mar 4 2017, 4:09 PM · ZFS

Mar 2 2017

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Optimize zone processing

Mar 2 2017, 12:59 PM · ZFS
slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

per-CPU cache must be drained before zone drain

Mar 2 2017, 11:53 AM · ZFS

Mar 1 2017

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

fix typo again

Mar 1 2017, 9:11 AM · ZFS
slw_zxy.spb.ru updated D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
Mar 1 2017, 8:43 AM · ZFS
slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

fix typo

Mar 1 2017, 8:42 AM · ZFS
slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  1. Check size of free items in zones (now eleminate false pressure to ARC by per-CPU zone cache)
  2. Do cleanup per-CPU zone cache after arc_kmem_reap_now() (immediatle available as free memory shrinked ARC cache)
Mar 1 2017, 8:40 AM · ZFS

Feb 26 2017

avg added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

If anyone is curious, here is a real world example of a deadlock involving the page daemon sleeping in ARC vm_lowmem hook.

48240 102111 aiod27         aiod27               stack8 sched_switch+0x143 mi_switch+0x194 sleepq_wait+0x42 _sleep+0x3c5 vm_wait+0x75 uma_small_alloc+0x6a keg_alloc_slab+0x9b keg_fetch_slab+0xdf zone_fetch_slab+0x42 zone_import+0x34 zone_alloc_item+0x3d keg_alloc_slab+0x2ba keg_fetch_slab+0xdf zone_fetch_slab+0x42 zone_import+0x34 uma_zalloc_arg+0x2fc arc_get_data_buf+0x489 arc_buf_alloc+0xc6 arc_read+0x121 dbuf_read+0x5a7 dbuf_findbp+0x183 dbuf_hold_impl+0x186 dbuf_hold+0x1b dmu_buf_hold_array_by_dnode_line+0x21b dmu_buf_hold_array+0x59 dmu_read_uio+0x3f zfs_freebsd_read+0x6d9 VOP_READ_APV+0x6e vn_read+0xca vn_io_fault+0x159 aio_daemon+0x991 fork_exit+0x11f fork_trampoline+0xe
   37 100626 zfskern        arc_reclaim_thread   stack41 sched_switch+0x143 mi_switch+0x194 sleepq_wait+0x42 _sx_xlock_hard+0x56f _sx_xlock+0x7e arc_buf_remove_ref+0x8c dbuf_rele_and_unlock+0x10e dbuf_evict+0x11 dbuf_do_evict+0x56 arc_do_user_evicts+0xb4 arc_reclaim_thread+0x222 fork_exit+0x11f fork_trampoline+0xe
    7 100113 pagedaemon     pagedaemon           stack49 sched_switch+0x143 mi_switch+0x194 sleepq_wait+0x42 _sx_xlock_hard+0x56f _sx_xlock+0x7e arc_lowmem+0x3c vm_pageout+0x20b fork_exit+0x11f fork_trampoline+0x
Feb 26 2017, 5:54 PM · ZFS

Feb 14 2017

slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#198159, @avg wrote:
  • there needs to be a very good explanation of why we would want to calculate needfree in the proposed fashion

main goal: don't allow drop freemem below v_free_min

I think that the existing check handles that already.

Feb 14 2017, 2:55 PM · ZFS
avg added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  • there needs to be a very good explanation of why we would want to calculate needfree in the proposed fashion

main goal: don't allow drop freemem below v_free_min

Feb 14 2017, 2:42 PM · ZFS
slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  • there needs to be a very good explanation of why we would want to calculate needfree in the proposed fashion
Feb 14 2017, 2:37 PM · ZFS
slw_zxy.spb.ru added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
In D7538#198085, @avg wrote:
  • I don't think that the needfree calculations in the patch are correct
    • in illumos needfree means something entirely different from what's calculated in the patch
    • there needs to be a very good explanation of why we would want to calculate needfree in the proposed fashion
Feb 14 2017, 2:01 PM · ZFS
avg added a comment to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
  • I don't think that the needfree calculations in the patch are correct
    • in illumos needfree means something entirely different from what's calculated in the patch
    • there needs to be a very good explanation of why we would want to calculate needfree in the proposed fashion
  • I have no objections against the part of the patch which comes from illumos, but then we should just import that change (7504 kmem_reap hangs spa_sync and administrative tasks)
  • I am in favor of removing cv_wait from the ARC lowmem hook
Feb 14 2017, 1:24 PM · ZFS

Feb 13 2017

slw_zxy.spb.ru added inline comments to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
Feb 13 2017, 8:47 PM · ZFS
allanjude added inline comments to D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
Feb 13 2017, 7:42 PM · ZFS
allanjude added reviewers for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD: mav, mahrens, pjd, avg, jpaetzel.
Feb 13 2017, 7:41 PM · ZFS
slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.
Feb 13 2017, 7:27 PM · ZFS

Aug 17 2016

slw_zxy.spb.ru updated the diff for D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD.

Illumos update needfree by kmem susbsystem. On FreeBSD emulate this in next way:

Aug 17 2016, 1:44 PM · ZFS

Aug 16 2016

slw_zxy.spb.ru retitled D7538: Correct adaptation ZFS ARC memory pressure to FreeBSD from to Correct adaptation of needfree (from illumos) to FreeBSD (ZFS ARC memory pressure).
Aug 16 2016, 9:59 PM · ZFS

Jul 28 2016

dnelson_1901_yahoo.com added a watcher for ZFS: dnelson_1901_yahoo.com.
Jul 28 2016, 4:55 AM

Jul 19 2016

mahrens added a member for ZFS: mahrens.
Jul 19 2016, 8:24 PM

Jun 10 2016

sean_coreitpro.com added a member for ZFS: sean_coreitpro.com.
Jun 10 2016, 9:11 PM

Jan 2 2016

eadler changed the edit policy for ZFS.
Jan 2 2016, 6:27 PM