So normally you'd set allow_sysctl_side_effects in the test metadata and then the test would automatically enable ktls (or be skipped because the test was run without it).

This has the disadvantage that you need to run kyua -v test_suites.FreeBSD.allow_sysctl_side_effects=1 test each time, even if ktls is already enabled, I think. OTOH, anyone running the test suite as part of a CI system would only need to specify that flag, and they wouldn't have to modify the host sysctl.conf.

IMO it'd be good to have a comment explaining roughly what this is testing for each cipher mode. It's a bit difficult to follow the macro expansions.

Hmm, I had copied this from how the POSIX AIO tests handle the "safe" knob which works without needing the verbose -v option. I wonder if there's a way to tell CI to flip the knob for the test but still let it be sane when run standalone?

Our CI sets a "ci" configuration variable that can fetched to determine if the test is running in the CI context.

I suppose we could have logic here to fetch the value of allow_sysctl_side_effects, and toggle the sysctl value if it's set, otherwise skip the test. Typically tests will include metadata which declares whether or not they need to be able to toggle sysctls, but I believe that metadata has to be fixed at compile time.

In any case, if the AIO tests already do something like this then I don't object. It would be nice if our tests were more consistent with this kind of things (whether or not to load KLDs is another one), but it's not worth blocking the review over.

Our CI sets a "ci" configuration variable that can fetched to determine if the test is running in the CI context.

I suppose we could have logic here to fetch the value of allow_sysctl_side_effects, and toggle the sysctl value if it's set, otherwise skip the test. Typically tests will include metadata which declares whether or not they need to be able to toggle sysctls, but I believe that metadata has to be fixed at compile time.

In any case, if the AIO tests already do something like this then I don't object. It would be nice if our tests were more consistent with this kind of things (whether or not to load KLDs is another one), but it's not worth blocking the review over.

I agree consistency would be good. Does CI always set allow_sysctl_side_effects or does it only set that if a test says it requires it in its require.config? If the former, then I think I would like what you propose of force-enabling it in the test if the setting is present. Might be nice to have a helper function for this perhaps and AIO could be updated to do this as well.

I suppose I could make this into a standalone function that takes the various values as arguments, or alternatively, I could push some of the logic such as setting up the tls_enable structures into test_ktls_transmit_app_data() and test_ktls_transmit_control(). This would make the macro-generated functions one-liners. If that is more readable I can take that approach. It is a bit of a shame one has to use the C pre-processor to generate multiple test cases rather than being able to use, say, a C for loop over an array of structs to declare the tests and run them.