- On illumos, needfree is part of the VM subsystem, which ZFS just observes.
FreeBSD emulate this. Current emulations is ugly: on pressure event needfree set to btoc(arc_c >> arc_shrink_shift) and don't change until arc_size drop below arc_c.
Until ARC reclaimed arc_c droped to arc_c_min, caused more and more reclaimed if arc_size fast enough reclaimed.
As result ZFS ARC size can be dramaticaly reduced, up to arc_min.
I can see 10x to 100x times needless arc reclaim.
- arc_shrink() hold freed memory in zone cache, this memory not available to system. Call arc_kmem_reap_now() for this.
- arc_kmem_reap_now() don't cleanup per-CPU zone cache. Implement uma_reclaim_zone_cache() and arc_drain_uma_cache() for this
- Many free memory immediatly available to ARC can be found in zone cache, check this for when call arc_reclaim_needed() for eleminate false memory pressure to ARC
Many thanks to markj, avg and mav for help, support and clarifications!
Tests and feedbacks welcome!