Page MenuHomeFreeBSD
Differential D55579

splice: optionally limit worker queues
Needs ReviewPublic
Actions

Authored by gallatin on Sat, Feb 28, 12:27 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Mar 3, 10:38 AM
Unknown Object (File)
Sun, Mar 1, 12:10 PM
Unknown Object (File)
Sun, Mar 1, 12:09 PM
Unknown Object (File)
Sun, Mar 1, 11:55 AM
Unknown Object (File)
Sun, Mar 1, 11:55 AM
Unknown Object (File)
Sun, Mar 1, 9:23 AM
Unknown Object (File)
Sat, Feb 28, 9:41 PM
Unknown Object (File)
Sat, Feb 28, 4:02 PM
View All 10 Files
Subscribers
imp
olce

Details

Reviewers
markj
glebius
Summary

This change introduces a tunable (kern.ipc.splice.num_wq) which can be used to limit the number
of splice worker queues. This can be desirable if you do not want splice threads to be able to
consume all available CPU on a server.

The default (-1) keeps the current behavior of running one worker for each core in the system.
An administrator can set it to 0 (either via tunable, or before the first splice call via sysctl) to
effectively disable splice, or some number smaller than the number of cores to limit
splice thread use.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

gallatin created this revision.Sat, Feb 28, 12:27 AM
Herald added subscribers: olce, imp. · View Herald TranscriptSat, Feb 28, 12:27 AM
gallatin requested review of this revision.Sat, Feb 28, 12:27 AM
imp added inline comments.Sat, Feb 28, 1:37 AM
sys/kern/uipc_socket.c
346 ↗(On Diff #172904)

This doesn't look like a tuneable. It's set to something else in both code paths.

gallatin updated this revision to Diff 172906.Sat, Feb 28, 1:48 AM
  • fixed typo in sysctl declaration
gallatin marked an inline comment as done.Sat, Feb 28, 1:49 AM
gallatin added inline comments.
sys/kern/uipc_socket.c
346 ↗(On Diff #172904)

Whoops, cut & paste bug. Good catch!

gallatin updated this revision to Diff 173093.Wed, Mar 4, 12:37 AM
gallatin marked an inline comment as done.
gallatin edited the summary of this revision. (Show Details)
markj added inline comments.Wed, Mar 4, 12:51 AM
FreeBSD/sys/kern/uipc_socket.c
1688

This won't work properly in the case where the CPU ID space has holes. That's what the loop was trying to handle before.

It might be easiest to rework the mapping between workers and CPU IDs: make splice_wq[] an array of mp_ncpus elements, and just use a counter instead of the CPU ID in the loop in splice_init() which starts each worker.

gallatin added inline comments.Wed, Mar 4, 2:03 AM
FreeBSD/sys/kern/uipc_socket.c
1688

Yeah, I had that in the first iteration, but backed it when I changed to starting all threads. I'll restore it.
Since we don't bind the worker to a CPU, my feeling is that this should all just be based on the number of cores..

gallatin updated this revision to Diff 173129.Wed, Mar 4, 2:56 PM

Re-worked thread setup to use mp_ncpus rather than mp_maxid, as suggested by @markj

gallatin marked an inline comment as done.Wed, Mar 4, 2:56 PM

Revision Contents
Changeset List

PathSize
FreeBSD/
sys/
kern/
50 lines

Diff 173129

View Options

FreeBSD/sys/kern/uipc_socket.c

Loading...