Page MenuHomeFreeBSD

nvmecontrol: New commands to support Fabrics hosts
ClosedPublic

Authored by jhb on Apr 9 2024, 11:03 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Jun 13, 6:27 PM
Unknown Object (File)
Sun, Jun 9, 12:02 PM
Unknown Object (File)
Thu, Jun 6, 11:37 PM
Unknown Object (File)
Mon, Jun 3, 8:20 AM
Unknown Object (File)
Mon, May 27, 2:11 AM
Unknown Object (File)
May 21 2024, 4:37 PM
Unknown Object (File)
May 4 2024, 10:37 PM
Unknown Object (File)
May 2 2024, 1:08 PM
Subscribers

Details

Summary
  • discover: Connects to a remote Discovery controller, fetches its Discovery Log Page, and enumerates the remote controllers described in the log page.

    The -v option can be used to display the Identify Controller data structure for the Discovery controller. This is only really useful for debugging.
  • connect: Connects to a remote I/O controller establish an association of an admin queue and a single I/O queue. The association is handed off to the in-kernel host to create a new nvmeX device.

    The -d option can be used to connect to a Discovery controller and attempt to create an association with each I/O controller enumerated in the Discovery controller's Discovery Log Page.
  • reconnect: Establishes a new association with a remote I/O controller for an existing nvmeX device. This can be used to restore access to a remote I/O controller after the loss of a prior association due to a transport error, controller reboot, etc.
  • disconnect: Deletes an nvmeX device after detaching its namespaces and terminating any active association.

Sponsored by: Chelsio Communications

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

jhb requested review of this revision.Apr 9 2024, 11:03 PM

Generally I like it.
I'd have been tempted to have a fabric command and these as subcommands. Did you consider this and if so what was your thinking for taking this path. And if not, what do you think of the idea.

sbin/nvmecontrol/connect.c
76

what's missing here?

96

what's your thinking for continuing after this, here and elsewhere?

In D44715#1021069, @imp wrote:

Generally I like it.
I'd have been tempted to have a fabric command and these as subcommands. Did you consider this and if so what was your thinking for taking this path. And if not, what do you think of the idea.

The current usage is somewhat akin to nvme(1) on Linux which has top-level connect, discover, etc. commands. The options passed after the command name are different, but the commands are named the same. It's also true though that Linux's nvme(1) doesn't have any subcommands (e.g. it has detach-ns instead of ns detach). Linux doesn't use connect-fabric though, it is just connect and connect-all (the latter similar to connect -d for nvmecontrol(8)).

In D44715#1021543, @jhb wrote:
In D44715#1021069, @imp wrote:

Generally I like it.
I'd have been tempted to have a fabric command and these as subcommands. Did you consider this and if so what was your thinking for taking this path. And if not, what do you think of the idea.

The current usage is somewhat akin to nvme(1) on Linux which has top-level connect, discover, etc. commands. The options passed after the command name are different, but the commands are named the same. It's also true though that Linux's nvme(1) doesn't have any subcommands (e.g. it has detach-ns instead of ns detach). Linux doesn't use connect-fabric though, it is just connect and connect-all (the latter similar to connect -d for nvmecontrol(8)).

OK. That makes some sense...

Is there a reason there's a different command line structure? The newer nvmecontrol commands I've implemented I've tried to be as close to Linux as is reasonable to make it easier for us to cope with vendor recipes... Though not always, so it's kinda a hodge podge...

This revision is now accepted and ready to land.Apr 16 2024, 7:25 PM

Move nvmf_transport_type to libnvmf

This revision now requires review to proceed.Apr 16 2024, 8:45 PM
In D44715#1021587, @imp wrote:
In D44715#1021543, @jhb wrote:
In D44715#1021069, @imp wrote:

Generally I like it.
I'd have been tempted to have a fabric command and these as subcommands. Did you consider this and if so what was your thinking for taking this path. And if not, what do you think of the idea.

The current usage is somewhat akin to nvme(1) on Linux which has top-level connect, discover, etc. commands. The options passed after the command name are different, but the commands are named the same. It's also true though that Linux's nvme(1) doesn't have any subcommands (e.g. it has detach-ns instead of ns detach). Linux doesn't use connect-fabric though, it is just connect and connect-all (the latter similar to connect -d for nvmecontrol(8)).

OK. That makes some sense...

Is there a reason there's a different command line structure? The newer nvmecontrol commands I've implemented I've tried to be as close to Linux as is reasonable to make it easier for us to cope with vendor recipes... Though not always, so it's kinda a hodge podge...

Hmm, I'll see how close I can get it perhaps. Linux calls connect-d connect-all and also has a disconnect-all. Some differences will be inherent I think since Linux doesn't have new-bus so their disconnect can't be the same, but I will see if I can more closely align them.

Rework command line options to more closely match nvme(1) on Linux

  • Split connect into connect (single controller) and connect-all (discover and connect multiple)
  • Update disconnect to support disconnecting by NQN and not just controller name. This now makes use of the NVMF_DISCONNET_HOST instead of using libdevctl directly. This also ensures it can't accidentally detach a PCI-e controller.
  • Add a disconnect-all command.
  • Update various flags to connect, etc. to match nvme(1) on Linux such as -g/-G for TCP data digests. Also add more flags matching Linux for number of I/O queues, queue sizes, keep alive timeout, and the host NQN.

Thanks for the rework!
I'm just finishing a nvme-cli port :)

This revision is now accepted and ready to land.May 1 2024, 10:15 PM

Switch to SPDX-only license blocks for C files

This revision now requires review to proceed.May 1 2024, 10:51 PM
This revision was not accepted when it landed; it landed in state Needs Review.May 3 2024, 12:16 AM
This revision was automatically updated to reflect the committed changes.