HomeFreeBSD

zvol: Support blk-mq for better performance

Description

zvol: Support blk-mq for better performance

Add support for the kernel's block multiqueue (blk-mq) interface in
the zvol block driver. blk-mq creates multiple request queues on
different CPUs rather than having a single request queue. This can
improve zvol performance with multithreaded reads/writes.

This implementation uses the blk-mq interfaces on 4.13 or newer
kernels. Building against older kernels will fall back to the
older BIO interfaces.

Note that you must set the zvol_use_blk_mq module param to
enable the blk-mq API. It is disabled by default.

In addition, this commit lets the zvol blk-mq layer process whole
struct request IOs at a time, rather than breaking them down
into their individual BIOs. This reduces dbuf lock contention
and overhead versus the legacy zvol submit_bio() codepath.

sequential dd to one zvol, 8k volblocksize, no O_DIRECT:

legacy submit_bio() 292MB/s write 453MB/s read
this commit 453MB/s write 885MB/s read

It also introduces a new zvol_blk_mq_chunks_per_thread module
parameter. This parameter represents how many volblocksize'd chunks
to process per each zvol thread. It can be used to tune your zvols
for better read vs write performance (higher values favor write,
lower favor read).

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ahelenia ZiemiaƄska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #13148
Issue #12483

Details

Provenance
Tony Hutter <hutter2@llnl.gov>Authored on Jun 9 2022, 2:10 PM
GitHub <noreply@github.com>Committed on Jun 9 2022, 2:10 PM
Parents
rG985c33b132f6: Introduce BLAKE3 checksums as an OpenZFS feature
Branches
Unknown
Tags
Unknown

Event Timeline

GitHub <noreply@github.com> committed rG6f73d02168ea: zvol: Support blk-mq for better performance (authored by Tony Hutter <hutter2@llnl.gov>).Jun 9 2022, 2:10 PM