Page MenuHomeFreeBSD

Optimize swp_pager_meta_lookup() to find additionally the numbers of continuous blocks.
Needs ReviewPublic

Authored by ota_j.email.ne.jp on Nov 16 2019, 2:39 PM.

Details

Reviewers
alc
kib
dougm
Summary

This is to reduce PCTRIE lookups from swap_pager_haspage().
Given swap_pager_haspage() is called frequently during heavy swap in and out activities, avoiding each individual lookup helps a lot.
For example, on i386, SWB_NPAGES is 32 and each swap_pager_haspage may call
PCTRIE_LOOKUP up to 65 times. Instead, new code will call 9 lookups for
the same size giving speed up of 700%.
On amd64, it is 64 and even twice faster as SWB_NPAGES is 64.

Test Plan

This changeset contains self-validations as a test driver.
I haven't seen any behavior changes between old and new implementations.
I will upload a version without self-test later.

make buildworld -j <hight number> causes lots of page in/out and is a
good test case

Diff Detail

Repository
rS FreeBSD src repository
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 27705
Build 25905: arc lint + arc unit

Event Timeline

Dropped self-test.

Updating D22409: swp_pager_meta_lookup() to find swap address and additionally search for

the number of continuous blocks.

Update and merge D22437.

Updating D22409: swp_pager_meta_lookup() to find swap address and additionally search for

the number of continuous blocks.

ota_j.email.ne.jp retitled this revision from swp_pager_meta_lookup() to find swap address and additionally search for the number of continuous blocks. to Optimize swp_pager_meta_lookup() to find additionally the numbers of continuous blocks..Nov 20 2019, 1:45 AM
ota_j.email.ne.jp edited the summary of this revision. (Show Details)
sys/vm/swap_pager.c
2163

I need to restore this KASSERT.

Restore KASSERT.

kib added a comment.Nov 21 2019, 12:30 PM

Where is this optimization actually used ? Almost all calls to vm_pager_has_page() pass NULLs for behind/ahead.

sys/vm/swap_pager.c
2163

before != NULL, there and in all other places

2164

How could this condition (left part before ||) be true. And even if it is, why do we care ?

sys/vm/swap_pager.c
1076

These swp_pager_meta_lookup function calls here and below are meld into a single swp_pager_meta_lookup. For up to the size of SWAP_META_PAGES, swp_pager_meta_lookup can access adjacent elements via array access avoiding most of PCTRIE lookup.

2164

The argument is an integer and vm_pindex_t is unsigned long long; I wanted to check negative values earlier to avoid comparing these 2 different types.

Check NULL pointers explicitly.