Page MenuHomeFreeBSD

Allow ZFS ARC min / max to be tuned at runtime
ClosedPublic

Authored by smh on Apr 11 2016, 9:09 AM.
Tags
None
Referenced Files
Unknown Object (File)
Feb 27 2024, 10:26 AM
Unknown Object (File)
Feb 27 2024, 10:13 AM
Unknown Object (File)
Feb 13 2024, 8:58 AM
Unknown Object (File)
Dec 20 2023, 12:57 PM
Unknown Object (File)
Dec 20 2023, 12:45 AM
Unknown Object (File)
Nov 1 2023, 9:49 PM
Unknown Object (File)
Oct 29 2023, 9:20 AM
Unknown Object (File)
Oct 20 2023, 11:12 AM

Details

Summary

Allow ZFS ARC min / max to be tuned at runtime

Test Plan
  • Decrease vfs.zfs.arc_max testing threshold values
  • Increase vfs.zfs.arc_max testing threshold values
  • Decrease vfs.zfs.arc_min testing threshold values
  • Increase vfs.zfs.arc_min testing threshold values

In each of the cases confirm ARC values adjust and memory usage follows.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

smh retitled this revision from to Allow ZFS ARC min / max to be tuned at runtime.
smh updated this object.
smh edited the test plan for this revision. (Show Details)
smh added reviewers: delphij, avg.

Remove redundent check on <= 0

allanjude added inline comments.
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
938 ↗(On Diff #15069)

It might be worth calling arc_kmem_reap_now() or kmem_reap() if the user is lowing the arc_max (as they obviously want memory back)

This cleans up and usually results in lowering the amount of 'wired' memory.

I am going to quickly experiment with that and get back to you.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
938 ↗(On Diff #15069)

When I tested it cleaned up quickly without that, so not sure its actually needed.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
938 ↗(On Diff #15069)

In my case it did not:

Frequently read files in ARC:

before:

Mem: 90M Active, 25M Inact, 61G Wired, 64G Free
ARC: 22G Total, 21G MFU, 555M MRU, 32K Anon, 173M Header, 67M Other

So I lowered the arc_max:
vfs.zfs.arc_max: 24000000000 -> 16000000000

Mem: 90M Active, 25M Inact, 61G Wired, 64G Free
ARC: 15G Total, 14G MFU, 555M MRU, 32K Anon, 169M Header, 49M Other

So the ARC shrunk, but no additional memory was freed

However if I run some sysctl's I added:
debug.zfs_arc_reap: 0 -> 0

Mem: 125M Active, 26M Inact, 38G Wired, 87G Free
ARC: 15G Total, 14G MFU, 555M MRU, 32K Anon, 169M Header, 50M Other

A great deal of the 'Wired' memory moves to free.

This does even more:
debug.zfs_kmem_reap: 0 -> 0

Mem: 90M Active, 25M Inact, 17G Wired, 107G Free
ARC: 15G Total, 14G MFU, 555M MRU, 32K Anon, 169M Header, 50M Other

It is not clear to me why in arc_kmem_reap_now() we only call kmem_reap() in the i386 case.

So I think we should either make lowering the arc_max call this, or, make it a separate sysctl similar to the temporary one I created. It is not instantaneous like changing arc_max is, so maybe separate makes more sense.

I just thought I'd mention this here, since usually the point of lowering arc_max is to free memory for another purpose.

allanjude edited edge metadata.

Tested on two of my machines. Works as expected.

Would love to see this in 11.0-RELEASE

This revision is now accepted and ready to land.Jun 28 2016, 11:28 PM
This revision was automatically updated to reflect the committed changes.