Lock avoiding pageout wakeup algorithm

[jeff]

Summary

This closes a few races in low memory conditions while avoiding locks by using the same threshold based notification algorithm I employed in the buffer cache. The algorithm uses atomics for free_count adjustment and then only threads which cross important thresholds attempt to check for waiters and wakeup. It can devolve into the same contention if you constantly bounce around the thresholds but this doesn't seem to be a problem in practice.

I improved the annotation and description of the locking protocol so hopefully this is fairly obvious from the code. I moved the pageout sleep/wakeup protection into its own locks to continue to shrink the scope of the free lock. I have a follow on patch that does the same for the allocation side but it is significantly larger.

[markj]

It's somewhat hard to read the diff without context. If you're uploading a raw patch, could you generate the patch with -U9999 or so?

vm/vm_page.c

2715

Style: the first letter should be capitalized and there should be a newline before the comment.

2724

Semicolon instead of a period.

vm/vm_pageout.c

1795

Typo in "we".

1823

"spurious"

1986

We have "curproc" has shorthand for curthread->td_proc.

vm/vm_pagequeue.h

273

Shouldn't we assert that the domain lock is held here? I would also assert that adj <= vmd_free_count.

[kib]

vm/vm_page.c

143

I think it is better to move the comment one line below.

736

int cast is excessive.

[jeff]

vm/vm_pagequeue.h

273

The domain lock is no longer required here. Right now it's only required between domain_available and freecount_dec. I have another patch that merges them into domain_allocate and does so with atomics.