pNFS client concurrent mirror I/O patch
ClosedPublic
Actions

Authored by rmacklem on Oct 9 2017, 8:15 PM.

Details

Reviewers: None
Commits: rS324676: Use taskqueue(9) to do writes/commits to mirrored DSs concurrently.

Summary

When the pNFS client is handling mirrored DS servers via the Flexible
File Layout, it must do writes and commits to all the mirrors.
This patch makes all but the last write/commit to a mirror be done
by a set of threads in a taskqueue, so that the RPCs will be done
concurrently.

Since changing the # of threads dynamically can't be done with
taskqueue, the # of threads is a sysctl which defaults to 32.
I think this is ok, since NFS I/O activity is so bursty that previous
I/O activity doesn't allow for good prediction of when threads
will be needed. As such, a fixed # of threads seems reasonable.

Test Plan

I have been testing this code against the pNFS server in
subversion under projects/pnfs-planb-server and it seems
to work ok. I do not know what the overheads of using
a taskqueue is, but I suspect it is less that doing a kproc_create()/kproc_exit()
for every RPC.

Diff Detail

Repository

rS FreeBSD src repository - subversion

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

rmacklem created this revision.Oct 9 2017, 8:15 PM

Herald added a subscriber: imp. · View Herald TranscriptOct 9 2017, 8:15 PM

The previous patch had a fundamental flaw, which was the code slept
waiting on completion of each RPC, serializing them.
This version of the patch doesn't sleep waiting for completion until
after all the RPCs have been started.

This variant of the patch sets the default # of threads to 8 * mp_ncpus instead of
a fixed 32. The sysctl allows a sysadmin to override this default.
Suggested by Julian Eischler.

Closed by commit rS324676: Use taskqueue(9) to do writes/commits to mirrored DSs concurrently. (authored by rmacklem). · Explain WhyOct 16 2017, 11:28 PM

This revision was automatically updated to reflect the committed changes.