Page MenuHomeFreeBSD

vfs: increment numvnodes without the vnode list lock unless under pressure
ClosedPublic

Authored by mjg on Jan 12 2020, 1:19 AM.

Details

Summary

The entire thing needs to be reworked. Either the code needs to be reworked to integrate it better with UMA (in particular so that it knows how to reclaim free vnodes) OR it needs to rely on the allocator to a lesser degree. I have a WIP patch for the former case which adds a callback to UMA.

This depends on D23158

In the meantime:

The vnode list lock is only needed to reclaim free vnodes or kick the vnlru thread (or to block and not miss a wake up (but note the sleep has a timeout so this would not be a correctness issue)). Try to get away without the lock by just doing an atomic increment.

The lock is contended e.g., during poudriere -j 104 where about half of all acquires come from vnode allocation code.

I can change this to a fcmpset loop if fetchadd seems too lax. While I don't expect the code to stay in this form in the long run, the benefit is that the change is eligible to be MFCed to stable/12.

Test Plan

sanity-wise buildkernel with kern.maxvnodes=5000 shows similar reclamation stats.

stress2

Diff Detail

Repository
rS FreeBSD src repository
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 28614

Event Timeline

mjg created this revision.Jan 12 2020, 1:19 AM
mjg updated this revision to Diff 66701.Jan 13 2020, 8:47 PM
mjg retitled this revision from vfs: increment numvnodes without the vnode free list lock unless under pressure to vfs: increment numvnodes without the vnode list lock unless under pressure.
mjg edited the summary of this revision. (Show Details)
mjg added a comment.Jan 15 2020, 11:43 PM

@kib @jeff any comments? the point is to lessen the load on freevnodes. right now it is an avoidable serialisation point for all vdrop/vhold which transition 0<->1 and reading it only when needed opens up options. apart from that vnode_lock is avoidably contended.

jeff added a comment.Jan 15 2020, 11:46 PM

I agree that this whole thing needs to be holistically reworked. I do not object to this patch but I would like kib to review.

kib accepted this revision.Jan 16 2020, 2:29 PM
This revision is now accepted and ready to land.Jan 16 2020, 2:29 PM