I would like to share my thoughts regarding this.

Currently, there is no accessible way to attach metadata to jails. This is commonly used elsewhere, for example in Kubernetes, to allow non-unique properties to enrich load balancers, schedulers, and volume provisioners to make more informed decisions.

Looks good to me too.

The email (https://lists.freebsd.org/archives/freebsd-testing/2024-November/000399.html) was understood to mean that you wanted to offer help with the development and were asking about a review. This was interpreted as a request to take over the project entirely, which is why I shared the full plan I intended to follow—to avoid prolonging the process, reduce the number of iterations, and clearly outline all expectations upfront. Please, let me know if you did not intend to work on this project and the email was misunderstood.

I see the currently proposed change as the following outcome:

• 13.x users should stay as is with old test suite and old Kyua from base or port/pkg. And the pkg is expected to be still cooked till 13 branch EOL.
• 14.x users are expected to find kyua pkg missing after upgrade to the latest port branch. They are expected to do such upgrade with OS upgrade as well, i.e. up to 14.2, which has the test suite aligned with Kyua in its base. 14.0 and 14.1 users of older Q branches are expected to use the same older suite and pkg. Even if they upgrade only ports having old 14.0 or 14.1 then they still have the test suite runnable with Kyua in their base.
• Just in case, the port is still available for anyone till 13 EOL (30 April 2026).
• A port/pkg update is expected to show the deprecation notice. It should cover attended cases.

IGNORE it for 14.0+

Thank you for working on this. If you plan to go ahead then please consider the following points.

I had to double check the things. I've delved into the details and discovered that the existing code does not allow to use CTLFLAG_PRISON as a flag which means that a variable varies per jail. I've stumbled upon the *allow* variables like these ones:

SYSCTL_PROC(_security_jail, OID_AUTO, set_hostname_allowed,
     CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE,
     NULL, PR_ALLOW_SET_HOSTNAME, sysctl_jail_default_allow, "I",
     "Processes in jail can set their hostnames (deprecated)");

Bump PORTREVISION with the expectation that users will see the deprecation notice upon upgrade.

Update the deprecation notice as suggested.

Please, check the updated patch as the follow-up of your email.

Covered by https://reviews.freebsd.org/D47334.

Covered by https://reviews.freebsd.org/D47105.

Another point is how to deal with sysctl which are "per jail" and vnet-related as well. I guess, they could end up with both flags?

This is the followup of https://reviews.freebsd.org/D47107.

Add description

This is the outcome of my analysis of the following CI test failure: https://ci.freebsd.org/view/Test/job/FreeBSD-main-amd64-test/25598/testReport/junit/sys.netpfil.pf/mbuf/inet6_in_mbuf_len/.
An extra 2001:db8::1 → 2001:db8::2 ICMPv6 Neighbor Advertisement 2001:db8::1 (sol) packet can appear. If it's sent after the net.dummymbuf.hits value is compared then it does not impact the results, but it may happen before.

This patch triggered the work on Kyua's execenv=jail feature and ended up being an example during the design discussion. As long as Mark's review comment to avoid exclusiveness in these tests has been recently resolved by introduction of Kyua's execenv=jail we can come back to the tests themselves. A separate fresh review is created for that: https://reviews.freebsd.org/D47163.

Use the standard way for error reporting

Now it looks like it does not require an extra comment, it runs in a jail to isolate from others and that's it. What do you think?

Move metadata from Kyuafile to the test itself

In D47105#1074583, @markj wrote:

Why exactly did this test need to be run in exclusive mode before? Some comments would be more useful.

Update ObsoleteFiles.inc respectively

It's planned to be used by pf and ipfw tests, the latter will be added with a separate patch.

@olivier , I remember this test was like a stumbling block for you in the past. I hope this patch does not impact your workflow. It's already based on jail usage and it seems logical to make it being jailed to avoid adding another exclusive test and not to do manual check & skip for the jail feature.

In D46684#1072028, @markj wrote:

Hrmph. Maybe we can just try to eliminate all uses of mtod(m, void *) then. I see some that are simply unnecessary, e.g., the ones in debugnet.c. Others can be converted to mtod(m, char *) or similar. What do you think?

In D46684#1071969, @markj wrote:

Do I understand correctly that casts to void * are the problematic cases? If so, can we do something clever with _Generic to avoid having to restrict consumers?

I've tried main kernel build for all make targets. I do not believe this is all the cases due to something can be hidden behind conditional preprocessing, but I guess it's the majority we could work with to reason our next steps (the full list can be found here):

Fix mtodo() macro

Fix style

Re-work to add assertions to the existing macros

According to the https://reviews.freebsd.org/D46684 it's agreed to work towards the global change of existing mtod() and mtodo().