Make swapoff reliable.

[kib]

Summary

swap_pager_swapoff() does trylock for pagein, which means that either io to md(4) over swap, or intensive page faults can prevent swapoff() from any progress. Then the retry < 100 check fails and machine panics.

If trylock fails, acquire the object lock in the blockable way and restart the hash bucket walk. I kept retries logic for now.

Another improvement there would be to check the ENOMEM conditions inside swap_pager_swapoff() before each page-in attempt. similar to the ENOMEM check in swapoff_one(), but this is non-trivial because swapoff_one() fills sw_blist, which must be undone on failure.

Test Plan

Peter has specific test that triggers the panic in swap_pager_swapoff(): failed to locate XXX swap blocks. The patched system survives it.

[markj]

This seems ok to me. Is there a specific reason to keep the retry logic?

sys/vm/swap_pager.c

1691 ↗

(On Diff #19782)

Missing period.

[kib]

In D7688#159797, @markj wrote:

This seems ok to me. Is there a specific reason to keep the retry logic?

One vague reason is that I am not completely convinced that retry per bucket logic is enough.

But more concrete thought is that this logic would allow for more reliability later. Please look at the swp_pager_force_pagein(). Right now this function simply panic if any error is returned by getpages(). By passing the error and retrying if needed we could improve the case.

On my previous job we did see "swap_pager: indefinite wait buffer" on heavily loaded machines.