While the change here is fine, converting ports en masse to TEST_TEST_TARGET may cause a regression for users who previously selected TEST option. Nothing currently depends on test target, so it has to be explicitly called for tests to run. Users shouldn't have to remember obscure make targets used by automation. regression-test suffered from this and became undertested after poudriere phased out tinderbox.

Better do-test should always be run like do-configure without HAS_CONFIGURE but implemented as a nop by default. To avoid incurring the cost of increased build time existing ports with tests that aren't using TEST option can hide behind DEVELOPER mode (e.g. poudriere bulk -t or poudriere testport). Optionally, DEVELOPER mode can enable TEST option by default but it'd require an exp-run.

While the change here is fine, converting ports en masse to TEST_TEST_TARGET

There will be no en masse TEST_TEST_TARGET, this artifact is only specific to ports for which dependencies or flags needed for testing should be installed before configure/build steps. But yes, DEVELOPER may imply WITH=TEST.

may cause a regression for users who previously selected TEST option

There will be no regression, as the tests should not have been run in the build process to start with. They were partially broken also.

Nothing currently depends on test target, so it has to be explicitly called for tests to run. Users shouldn't have to remember obscure make targets used by automation. regression-test suffered from this and became undertested after poudriere phased out tinderbox.

The plan is to add test support to poudriere, bdrewery@ should be working on this.

Better do-test should always be run like do-configure without HAS_CONFIGURE but implemented as a nop by default.

Always running target which is no op by default is just the more obscure version of not running a real target by default, and the latter is how make test works.

You don't need it in the normal port build process and package builds at all, as it slows these down, requires extra depends, and there are false positives, but users can run make test anytime, and maintainers/committers should always run it. Separate poudriere with enabled test can be set up too (there actually one which tests with DEVELOPER, it may run tests too http://package21.nyi.freebsd.org).