Differential D60473

nvmft: Add tests for various various shutdown/disconnect scenarios
Needs ReviewPublic
Actions

Authored by jhb on Thu, Oct 8, 3:51 PM.

Details

Reviewers

markj

Group Reviewers

cam

Summary

The nvmf_shutdown helper program connects to a remote NVMeoF controller and then shuts down the controller in various ways (reset vs shutdown vs connection close) with an optional delay between the shutdown and closing the connection.

If the initial shutdown is not a connection close, the program can also re-enable the controller (after an optional delay) before shutting down the controller again (using any of the available methods).

The shutdown tests require an address and NQN of a remote controller to be specified via configuration variables. To test FreeBSD's in-kernel controller, ctld must be running in the environment with the 'addr' and 'nqn' config variables set to values from the config file. A sample ctl.conf is included for test environments to use.

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Skipped

Unit

Tests Skipped

Build Status

Buildable 77988
Build 74871: arc lint + arc unit

Event Timeline

jhb created this revision.Thu, Oct 8, 3:51 PM

Herald added a reviewer: cam. · View Herald TranscriptThu, Oct 8, 3:51 PM

Herald added subscribers: ivy, imp. · View Herald Transcript

jhb requested review of this revision.Thu, Oct 8, 3:51 PM

Harbormaster completed remote builds in B77857: Diff 189064.Thu, Oct 8, 3:51 PM

jhb added a parent revision: D60472: nvmft: Cancel and drain any existing terminate task when enabling a controller.Thu, Oct 8, 3:51 PM

jhb added a child revision: D60474: nvmft: Validate the native byte order qid when establishing I/O queues.

jhb mentioned this in D60470: nvmft: Clear CSTS.SHST when a controller is enabled.Thu, Oct 8, 3:55 PM

jhb mentioned this in D60472: nvmft: Cancel and drain any existing terminate task when enabling a controller.

So for testing the bug fixed by D60472 I wrote the nvmf_shutdown tool. The tests in the shell script are not exhaustive in that the tool permits testing more combinations and various timeouts, etc. and I kept the tool in this version as I think it might be useful in the future for other tests? That said, this does require the somewhat messy logic to construct the shell scripts, etc. Contrast with other nvmft tests I've added that are all in C (D60478 and D60479). If I were to write these in C instead I could use C preprocessor magic similar to what I've done in the KTLS tests instead of the Makefile goo here.

Finally, the original version of this test tried to start/stop ctld for each test, but that's kind of unworkable and prevents running these tests in parallel (which is ideal). Instead, these tests accept the address and NQN of a controller to test as a set of variables and the tester is responsible for setting up an NVMe server to test against. For FreeBSD's CI my intention is that the ctl.conf I included in this directory can be used with ctld to start a suitable daemon, though I don't document the variables one would need to set in kyua.conf in that case.

jhb added a reviewer: markj.Thu, Oct 8, 4:04 PM

Hum, I just realized that there are a few tests which require that ctld not be running, see usr.sbin/ctladm/tests/port.sh. I guess there's not much we can do about that unless we go back to requiring each test start/stop ctld, which in turn makes them slow and non-parallelizable.

tests/sys/cam/ctl/nvmft/Makefile
46	Can we lift this out into a script? I also wonder if we should generate the scripts ahead of time. That's what I did for dtrace tests, in cddl/usr.sbin/dtrace/tests/tools/gentest.sh. I'm not sure if there is any particular advantage in this case since there aren't that many tests, but it might be nicer to avoid doing extra work in each build. And, it'll be easier to debug failing tests that way.