Page MenuHomeFreeBSD
Differential D14905

per-cpu free page caching
ClosedPublic
Actions

Authored by jeff on Mar 30 2018, 3:47 AM.
Tags
None
Referenced Files
Unknown Object (File)
Jan 21 2025, 11:48 PM
Unknown Object (File)
Jan 21 2025, 12:44 AM
Unknown Object (File)
Jan 8 2025, 11:56 PM
Unknown Object (File)
Jan 8 2025, 11:26 PM
Unknown Object (File)
Dec 22 2024, 10:16 PM
Unknown Object (File)
Dec 21 2024, 8:07 PM
Unknown Object (File)
Nov 21 2024, 6:43 AM
Unknown Object (File)
Nov 18 2024, 8:03 AM
View All 52 Files
Subscribers
imp

Details

Reviewers
alc
markj
kib
Commits
rS331863: Add a uma cache of free pages in the DEFAULT freepool. This gives us
Summary

This implements per-cpu free page caching only for FREEPOOL_DEFAULT. I haven't seen enough contention yet from FREEPOOL_DIRECT that it is an issue. This also brings in vm_phys_alloc_npages() which gives you as many as 'count' pages. This is an optimization to reduce round-trips to the backend phys allocator. Contrary to other instances of this patch, this is now completely compatible with reservations.

It might be more elegant to push this into vm_phys.c along with all of the vm_domain_free_lock() calls. However, pushing it there would mean the cached pages were reflected in the free count. I believe this will make us more prone to out of memory deadlocks.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

jeff created this revision.Mar 30 2018, 3:47 AM
Herald added a subscriber: imp. · View Herald TranscriptMar 30 2018, 3:47 AM
kib added inline comments.Mar 30 2018, 3:04 PM
vm/vm_page.c
224

Should it be sizeof(struct vm_page *) ?

jeff added inline comments.Mar 30 2018, 11:24 PM
vm/vm_page.c
224

actually in this case uma uses it as a cost function to limit the bucket size. But we override with MAXBUCKET. So it is correct to intent but unused.

markj accepted this revision.Apr 1 2018, 4:20 AM
This revision is now accepted and ready to land.Apr 1 2018, 4:20 AM
Closed by commit rS331863: Add a uma cache of free pages in the DEFAULT freepool. This gives us (authored by jeff). · Explain WhyApr 1 2018, 4:50 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
vm/
4 lines
8 lines
94 lines
2 lines
1 line
20 lines

Diff 40906

View Options

vm/uma.h

Show First 20 Lines • Show All 1,492 Lines • ▼ Show 20 Linesint ttm_bo_global_init(struct drm_global_reference *ref)
mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF); mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF);
glob->mem_glob = bo_ref->mem_glob; glob->mem_glob = bo_ref->mem_glob;
tries = 0; tries = 0;
retry: retry:
glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0, glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE); VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE);
if (unlikely(glob->dummy_read_page == NULL)) { if (unlikely(glob->dummy_read_page == NULL)) {
if (tries < 1 && vm_page_reclaim_contig(0, 1, 0, if (tries < 1 && (vm_page_reclaim_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0)) { VM_MAX_ADDRESS, PAGE_SIZE, 0) == 0)) {
tries++; tries++;
goto retry; goto retry;
} }
ret = -ENOMEM; ret = -ENOMEM;
goto out_no_drp; goto out_no_drp;
} }
INIT_LIST_HEAD(&glob->swap_lru); INIT_LIST_HEAD(&glob->swap_lru);
▲ Show 20 Lines • Show All 382 Lines • Show Last 20 Lines
View Options

vm/uma_core.c

Show First 20 Lines • Show All 1,492 Lines • ▼ Show 20 Linesint ttm_bo_global_init(struct drm_global_reference *ref)
mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF); mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF);
glob->mem_glob = bo_ref->mem_glob; glob->mem_glob = bo_ref->mem_glob;
tries = 0; tries = 0;
retry: retry:
glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0, glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE); VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE);
if (unlikely(glob->dummy_read_page == NULL)) { if (unlikely(glob->dummy_read_page == NULL)) {
if (tries < 1 && vm_page_reclaim_contig(0, 1, 0, if (tries < 1 && (vm_page_reclaim_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0)) { VM_MAX_ADDRESS, PAGE_SIZE, 0) == 0)) {
tries++; tries++;
goto retry; goto retry;
} }
ret = -ENOMEM; ret = -ENOMEM;
goto out_no_drp; goto out_no_drp;
} }
INIT_LIST_HEAD(&glob->swap_lru); INIT_LIST_HEAD(&glob->swap_lru);
▲ Show 20 Lines • Show All 382 Lines • Show Last 20 Lines
View Options

vm/vm_page.c

Show First 20 Lines • Show All 1,492 Lines • ▼ Show 20 Linesint ttm_bo_global_init(struct drm_global_reference *ref)
mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF); mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF);
glob->mem_glob = bo_ref->mem_glob; glob->mem_glob = bo_ref->mem_glob;
tries = 0; tries = 0;
retry: retry:
glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0, glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE); VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE);
if (unlikely(glob->dummy_read_page == NULL)) { if (unlikely(glob->dummy_read_page == NULL)) {
if (tries < 1 && vm_page_reclaim_contig(0, 1, 0, if (tries < 1 && (vm_page_reclaim_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0)) { VM_MAX_ADDRESS, PAGE_SIZE, 0) == 0)) {
tries++; tries++;
goto retry; goto retry;
} }
ret = -ENOMEM; ret = -ENOMEM;
goto out_no_drp; goto out_no_drp;
} }
INIT_LIST_HEAD(&glob->swap_lru); INIT_LIST_HEAD(&glob->swap_lru);
▲ Show 20 Lines • Show All 382 Lines • Show Last 20 Lines
View Options

vm/vm_pagequeue.h

Show First 20 Lines • Show All 1,492 Lines • ▼ Show 20 Linesint ttm_bo_global_init(struct drm_global_reference *ref)
mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF); mtx_init(&glob->lru_lock, "ttmlru", NULL, MTX_DEF);
glob->mem_glob = bo_ref->mem_glob; glob->mem_glob = bo_ref->mem_glob;
tries = 0; tries = 0;
retry: retry:
glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0, glob->dummy_read_page = vm_page_alloc_noobj_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE); VM_MAX_ADDRESS, PAGE_SIZE, 0, VM_MEMATTR_UNCACHEABLE);
if (unlikely(glob->dummy_read_page == NULL)) { if (unlikely(glob->dummy_read_page == NULL)) {
if (tries < 1 && vm_page_reclaim_contig(0, 1, 0, if (tries < 1 && (vm_page_reclaim_contig(0, 1, 0,
VM_MAX_ADDRESS, PAGE_SIZE, 0)) { VM_MAX_ADDRESS, PAGE_SIZE, 0) == 0)) {
tries++; tries++;
goto retry; goto retry;
} }
ret = -ENOMEM; ret = -ENOMEM;
goto out_no_drp; goto out_no_drp;
} }
INIT_LIST_HEAD(&glob->swap_lru); INIT_LIST_HEAD(&glob->swap_lru);
▲ Show 20 Lines • Show All 382 Lines • Show Last 20 Lines
View Options

vm/vm_phys.h

Loading...
View Options

vm/vm_phys.c

Loading...