Change Details

These both arise when multiple applications are consuming free pages at a high rate (e.g., > 1GB/s), and the inactive queue contains plenty of clean pages. 1. After completing an inactive queue scan, we might still be below the v_free_min threshold, in which case there are threads still sleeping in VM_WAIT. In this case we should not go to sleep - we should immediately start another scan. 2. vm_pageout_wanted may be set without the free queues lock held. This can lead to a race: - the page daemon checks vm_pageout_wanted in vm_pageout_worker() and sees that it's false. vm_pages_needed is false. - a thread sets vm_pageout_wanted without holding the queue lock - the page daemon goes to sleep thinking that it has no work to do - a thread enters VM_WAIT, sees that vm_pageout_wanted is true and thus does not wake up the page daemon - the page daemon sleeps for a full second before starting another scan, despite the fact that there are threads in VM_WAIT Fix this by adding a new subroutine, pagedaemon_wait(), called from vm_wait() and vm_waitpfault(). It wakes up the page daemon if either vm_pageout_wanted or vm_pages_needed is false, and atomically sleeps on v_free_count.

These both arise when multiple applications are consuming free pages at a high rate (e.g., > 1GB/s), and the inactive queue contains plenty of clean pages. 1. After completing an inactive queue scan, we might still be below the v_free_min threshold, in which case there are threads still sleeping in VM_WAIT. 2. vm_pageout_wanted may be set without the free queues lock held. This can lead to a race: - the page daemon checks vm_pageout_wanted in vm_pageout_worker() and sees that it's false. vm_pages_needed is false. - a thread sets vm_pageout_wanted without holding the queue lock - the page daemon goes to sleep thinking that it has no work to do - a thread enters VM_WAIT, sees that vm_pageout_wanted is true and thus does not wake up the page daemon - the page daemon sleeps for a full second before starting another scan, despite the fact that there are threads in VM_WAIT Fix 1) by ensuring that we immediately start a new inactive queue scan if v_free_count < v_free_min and threads are blocked in VM_WAIT. Fix 2) by adding a new subroutine, pagedaemon_wait(), called from vm_wait() and vm_waitpfault(). It wakes up the page daemon if either vm_pageout_wanted or vm_pages_needed is false, and atomically sleeps on v_free_count.

These both arise when multiple applications are consuming free pages at a high rate (e.g., > 1GB/s), and the inactive queue contains plenty of clean pages. 1. After completing an inactive queue scan, we might still be below the v_free_min threshold, in which case there are threads still sleeping in VM_WAIT. In this case we should not go to sleep - we should immediately start another scan. 2. vm_pageout_wanted may be set without the free queues lock held. This can lead to a race: - the page daemon checks vm_pageout_wanted in vm_pageout_worker() and sees that it's false. vm_pages_needed is false. - a thread sets vm_pageout_wanted without holding the queue lock - the page daemon goes to sleep thinking that it has no work to do - a thread enters VM_WAIT, sees that vm_pageout_wanted is true and thus does not wake up the page daemon - the page daemon sleeps for a full second before starting another scan, despite the fact that there are threads in VM_WAIT Fix this1) by addensuring a new subroutine, pagedaemon_wait(), called fromthat we immediately start a new inactive queue scan if v_free_count < v_free_min and threads are blocked in VM_WAIT. Fix 2) by adding a new subroutine, pagedaemon_wait(), called from vm_wait() and vm_waitpfault(). It wakes up the page daemon if either vm_pageout_wanted or vm_pages_needed is false, and atomically sleeps on v_free_count.