Page MenuHomeFreeBSD

nvmfd: A simple userspace daemon for the NVMe over Fabrics controller

Authored by jhb on Apr 9 2024, 11:05 PM.
Referenced Files
Unknown Object (File)
Sun, Jun 16, 9:43 AM
Unknown Object (File)
Sat, Jun 15, 8:23 PM
Unknown Object (File)
Thu, Jun 13, 6:28 PM
Unknown Object (File)
Mon, May 27, 2:12 AM
Unknown Object (File)
Wed, May 22, 11:51 PM
Unknown Object (File)
May 21 2024, 2:17 AM
Unknown Object (File)
May 15 2024, 7:33 PM
Unknown Object (File)
May 3 2024, 11:14 AM



This daemon can operate as a purely userspace controller exporting one
or more simulated RAM disks or local block devices as NVMe namespaces
to a remote host. In this case the daemon provides a discovery
controller with a single entry for an I/O controller.

nvmfd can also offload I/O controller queue pairs to the nvmft.ko
in-kernel Fabrics controller when -K is passed. In this mode, nvmfd
still accepts connections and performs initial transport-specific
negotitation in userland. The daemon still provides a userspace-only
discovery controller with a single entry for an I/O controller.
However, queue pairs for the I/O controller are handed off to the CTL
NVMF frontend.

Eventually ctld(8) should be refactored to to provide an abstraction
for the frontend protocol and the discovery and the kernel mode of
this daemon should be merged into ctld(8). At that point this daemon
can be moved to tools/tools/nvmf as a debugging tool (mostly as sample
code for a userspace controller using libnvmf).

Sponsored by: Chelsio Communications

Diff Detail

rG FreeBSD src repository
Lint Not Applicable
Tests Not Applicable

Event Timeline

jhb requested review of this revision.Apr 9 2024, 11:05 PM
jhb created this revision.

This seems good, but again, it's kinda huge and that makes careful review is hard.


Consider omitting the boilerplate, here and elsewhere in the reviews.


I'd make this a #define... it could be 4k too, or at least it might be nice.


what happens when malloc fails? core dump?

This revision is now accepted and ready to land.Apr 12 2024, 12:36 AM

I guess this can use DEV_BSIZE as that is hardcoded as 512 in practice.

In theory this could be a configurable knob, but the userspace controller isn't really intended for serious production use, more for testing / debugging.


Yep. (Again, this is only relevant when running the userspace controller, not for the production use case).

Update TCP digest command line options to match nvme(1) on Linux

This revision now requires review to proceed.May 1 2024, 9:13 PM

Switch to SPDX-only license blocks for C files

This revision was not accepted when it landed; it landed in state Needs Review.May 3 2024, 12:17 AM
This revision was automatically updated to reflect the committed changes.