The issue solved is that pausing before cleanup with -p or --pause-before-cleanup options of kyua debug command yields the following awkward sequence of events, where all the output is visible only when we resume the process:
root@taarch64:/usr/tests/sys/netpfil/pf # kyua debug --pause-before-cleanup mbuf:inet_in_mbuf_len The test paused right before its cleanup routine. Test work dir: /tmp/kyua.cBfB6F/2/work Press <Enter> to continue... Executing command [ ping -c1 192.0.2.2 ] Executing command [ ping -c1 -t1 192.0.2.2 ] Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: -> inet in epair0b pull-head 0; Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: inet in epair0b pull-head 0; -> inet in epair0b pull-head 1; net.dummymbuf.hits: 1 -> 0 Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: inet in epair0b pull-head 1; -> inet in epair0b pull-head 19; net.dummymbuf.hits: 1 -> 0 Executing command [ ping -c1 192.0.2.2 ] alcatraz: removed pf enabled Ethernet rules cleared rules cleared nat cleared 0 tables deleted. rules cleared 0 states cleared source tracking entries cleared pf: statistics cleared pf: interface flags reset Ethernet rules cleared rules cleared nat cleared 0 tables deleted. rules cleared 0 states cleared source tracking entries cleared pf: statistics cleared pf: interface flags reset ifconfig: interface epair0a does not exist mbuf:inet_in_mbuf_len -> passed
This is because of internally kyua runs the test using the same mechanism which is used by kyua test command, when stdout/stderr are written to plain files beneath /tmp/kyua.*/**/stdout and /tmp/kyua.*/**/stderr. And it's printed back to actual stdout/stderr only when the test is finished.
The proposed patch keeps the same behavior if -p|--pause-before-cleanup options are not used, but skips writing to temporary files if pausing mechanism is asked to be used. Moreover, kyua debug -p still supports kyua debug --stdout=<path> --stderr=<path> options.
As a result, we get the expected sequence in the output:
root@taarch64:/usr/tests/sys/netpfil/pf # kyua debug --pause-before-cleanup mbuf:inet_in_mbuf_len Executing command [ ping -c1 192.0.2.2 ] pf enabled Ethernet rules cleared rules cleared nat cleared 0 tables deleted. rules cleared 0 states cleared source tracking entries cleared pf: statistics cleared pf: interface flags reset Executing command [ ping -c1 -t1 192.0.2.2 ] Ethernet rules cleared rules cleared nat cleared 0 tables deleted. rules cleared 0 states cleared source tracking entries cleared pf: statistics cleared pf: interface flags reset Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: -> inet in epair0b pull-head 0; Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: inet in epair0b pull-head 0; -> inet in epair0b pull-head 1; net.dummymbuf.hits: 1 -> 0 Executing command [ ping -c1 192.0.2.2 ] net.dummymbuf.rules: inet in epair0b pull-head 1; -> inet in epair0b pull-head 19; net.dummymbuf.hits: 1 -> 0 Executing command [ ping -c1 192.0.2.2 ] The test paused right before its cleanup routine. Test work dir: /tmp/kyua.OYNAoB/2/work Press <Enter> to continue... alcatraz: removed ifconfig: interface epair0a does not exist mbuf:inet_in_mbuf_len -> passed
The kyua debug -x option (https://reviews.freebsd.org/D52642) could also support it after landing of this change.