Thank you for working on this!
Any thoughts of using native python interface instead of the shell wrappers?
Just in case, the examples can be found in tests/sys/net/routing/test_rtsock_multipath.py or tests/sys/netlink/test_rtnl_iface.py

In D37876#860539, @melifaro wrote:

Any thoughts of using native python interface instead of the shell wrappers?

That looks nice!, but do you mean using pytest instead of atf-sh(3), or instead of using subprocess.run to execute shell commands?
If the latter, given a separation between kernel and userland tests is likely desired, duplication of the tools/helpers is what we should do initially?
For example, instead of:

import subprocess

subprocess.run(["sysctl", "net.inet.ip.process_options=0"], check=True)

let's change it to a native python interface, using tests/atf_python/sys/net/tools.py:

from atf_python.sys.net.tools import ToolsHelper

ToolsHelper.set_sysctl("net.inet.ip.process_options", 0)

Should we duplicate tools.py initially, and place it under a common test directory for userland? Given downstream projects may consume userland- or kernel-only code from FreeBSD.
I was planning on stealing common code from tests/sys for a future VNET jails integration, so the placement of vnet.subr was going to raise the same question.

In D37876#860550, @jlduran_gmail.com wrote:

In D37876#860539, @melifaro wrote:

Any thoughts of using native python interface instead of the shell wrappers?

That looks nice!, but do you mean using pytest instead of atf-sh(3), or instead of using subprocess.run to execute shell commands?

Pytest. There is a wrapper that automatically wraps pytest tests into kyua interface, allowing to run them in a standard system test way via kuya.

If the latter, given a separation between kernel and userland tests is likely desired, duplication of the tools/helpers is what we should do initially?

In’s not exactly kernel/userland but more like just as include, so all of the files are good to use in any tests.

For example, instead of:

import subprocess

subprocess.run(["sysctl", "net.inet.ip.process_options=0"], check=True)

let's change it to a native python interface, using tests/atf_python/sys/net/tools.py:

from atf_python.sys.net.tools import ToolsHelper

ToolsHelper.set_sysctl("net.inet.ip.process_options", 0)

Absolutely.

Should we duplicate tools.py initially, and place it under a common test directory for userland? Given downstream projects may consume userland- or kernel-only code from FreeBSD.

I’d suggest using (and altering) the existing tools.py

I was planning on stealing common code from tests/sys for a future VNET jails integration, so the placement of vnet.subr was going to raise the same question.

I’d vote for just re-using it. Happy to review/accept any changes required

I think this looks really great. I assume you have a few extra test cases that you're planning to add to ping_test.sh? Personally, I don't have strong opinions about whether to use atf-sh or pytest, as long as you can get the cleanup and metadata correct. And don't forget that you'll need to add injection.py to ObsoleteFiles.mk, or else rename pinger.py to injection.py.

In D37876#860561, @asomers wrote:

I assume you have a few extra test cases that you're planning to add to ping_test.sh?

Yes! Absolutely, I currently have more than 10 tests using this script.

And don't forget that you'll need to add injection.py to ObsoleteFiles.mk, or else rename pinger.py to injection.py.

Thank you, I was completely unaware of this step.
There it goes my ping-themed series of tests... I even had the jails named BRL, after the Ballistic Research Lab, :-(
I'll think it over, but it is likely I won't pollute ObsoleteFiles.mk out of sheer vanity and rename the script. However, it can do more than just inject packets.

Ahh, you're correct about the cleanup directory. I misremembered that.

@asomers I can drop the jailing part at your request, of course, although I personally prefer to have these tests jailed.
I am also playing with the native python interface, it'll take me a while, since I already had everything set up for atf-sh. I stopped short of getting started with IPv6 tests, and now I'm grateful, as I am currently lured by the possibility of using parameterized pytests.