In swap_pager_unswapped, include an attempt to temporarily upgrade a read lock to an exclusive lock in order to edit the pctrie. Allow this to be invoked without the write lock held.
Unit Tests Skipped
Is this check still needed after the rest of the changes ?
Can you split this change and tryupgrade in swap_pager_unswapped into separate review ? I think that introduction of lookup() and removal of SWM_POP is obviously good change, but it should not be mixed with locking logic.
Given swp_pager_meta_ctl() with SWM_POP calls VM_OBJECT_ASSERT_WLOCKED, I expected VM_OBJECT_ASSERT_WLOCKED.
swp_pager_freeswapspace calls blist_free while swp_pager_free_empty_swblk calls uma_zfree.
I am somewhat curious how much does the upgrade succeed. Did you measured it ? E.g. the counters of total vm_pager_page_unswapped calls during buildworld vs. the number of times the lock needs to be upgraded vs. the number of successfull upgrades.
I measured it, and learned nothing. How should I test so that I might learn something.
World build completed on Mon Dec 9 04:03:03 CST 2019
World built in 6609 seconds, ncpu: 6, make -j6