Page MenuHomeFreeBSD

kyua: Add "debug -x|--execute cmd" option
AcceptedPublic

Authored by igoro on Sep 20 2025, 3:35 PM.
Tags
None
Referenced Files
F140918415: D52642.diff
Mon, Dec 29, 3:28 PM
Unknown Object (File)
Nov 25 2025, 2:07 PM
Unknown Object (File)
Nov 13 2025, 10:49 PM
Unknown Object (File)
Nov 13 2025, 8:38 AM
Unknown Object (File)
Nov 5 2025, 1:02 AM
Unknown Object (File)
Nov 3 2025, 10:59 AM
Unknown Object (File)
Oct 27 2025, 8:49 PM
Unknown Object (File)
Oct 26 2025, 7:26 AM
Subscribers

Details

Reviewers
markj
ngie
Group Reviewers
tests

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 67198
Build 64081: arc lint + arc unit

Event Timeline

igoro requested review of this revision.Sep 20 2025, 3:35 PM

Always attempt to use $SHELL

The prototype has the following logic currently:

1 Invoking w/o argument runs $SHELL (falls back to hardcoded /usr/bin/ if SHELL env var is not found). It does chdir to the test's working dir:

> kyua debug -x test1
> kyua debug --execute test1

root@taarch64:/usr/tests/sys/netpfil/pf # kyua debug -x mbuf:inet_in_mbuf_len
root@taarch64:/tmp/kyua.h6W16I/2/work # ls -a
.                       ..                      created_interfaces.lst  created_jails.lst
root@taarch64:/tmp/kyua.h6W16I/2/work # exit
Executing command [ ping -c1 192.0.2.2 ]
alcatraz: removed
Fail: incorrect exit status: 0, expected: 1
stdout:
PING 192.0.2.2 (192.0.2.2): 56 data bytes
64 bytes from 192.0.2.2: icmp_seq=0 ttl=64 time=0.097 ms

--- 192.0.2.2 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.097/0.097/0.097/0.000 ms

stderr:

Files left in work directory after failure: created_interfaces.lst, created_jails.lst
ifconfig: interface epair0a does not exist
mbuf:inet_in_mbuf_len  ->  failed: atf-check failed; see the output of the test for details
root@taarch64:/usr/tests/sys/netpfil/pf # echo $?
1

2 If an argument is passed then $SHELL (falls back to hard-coded /bin/sh) is run with the given command string (aka $SHELL -c $cmd). A shell is expected to have -c option. It also does chdir to the test's working dir:

> kyua debug -x'printf "\a"' test1
> kyua debug --execute="cat some_file_from_test_working_dir; do_something_else" test1

The default string is intentionally set to $SHELL literal to be a hint during help output reading:

root@taarch64:/usr/tests/sys/netpfil/pf # kyua help debug
kyua (Kyua) 0.13

Usage: kyua [general_options] debug [command_options] test_case

...

Available command options:
  --build-root=path                     Path to the built test programs, if different from the location of the
                                        Kyuafile scripts.
  -k file, --kyuafile=file              Path to the test suite definition (default: Kyuafile).
  -p, --pause-before-cleanup-upon-fail  Pauses right before the test cleanup upon fail.
  --pause-before-cleanup                Pauses right before the test cleanup.
  --stdout=path                         Where to direct the standard output of the test case (default: /dev/stdout).
  --stderr=path                         Where to direct the standard error of the test case (default: /dev/stderr).
  -x cmd, --execute=cmd                 A command to run upon test failure (default: $SHELL).

See kyua-debug(1) for more details.

P.S. Eventually, I plan to extract base_option/string_option arg optionality to separate commit(s).

This works almost exactly how I want, thank you. I have one "complaint": when I use this mode, kyua 1) drops me in the shell, 2) waits for the shell to exit, 3) prints the debug output, e.g., executed commands and the test failure string. The -p mode is basically the same. It is more useful to do 3) first, then 1) and 2). Is it hard to make this change?

This works almost exactly how I want, thank you. I have one "complaint": when I use this mode, kyua 1) drops me in the shell, 2) waits for the shell to exit, 3) prints the debug output, e.g., executed commands and the test failure string. The -p mode is basically the same. It is more useful to do 3) first, then 1) and 2). Is it hard to make this change?

Yes, both kyua debug -x and kyua debug -p have the same behavior as the same internal mechanism is used. The following patch addresses this "inconvenience by design": https://reviews.freebsd.org/D54363

Make it skip writing stdout/stderr to tmp files, as "debug -p" does

contrib/kyua/cli/cmd_debug.cpp
105

The next version of kyua should focus on more modern standards (C++-17 at least; ideally C++-20). With that in mind you can technically use std::filesystem_path::current_path(..) instead of chdir(2). I'm trying to wrangle everything together upstream since I have some cycles right now.
FWIW, I'm ok with using chdir(2) here (and with what goes into FreeBSD:main), but I would really like to focus on a better solution upstream. Please submit a solution (upstream) leveraging this API. We can reconcile this along with the other local changes in the upcoming release.

This works well, thanks. It would probably be useful to document/log the fact that if the test uses a execenv jail, then -x will attach the command to the jail.

contrib/kyua/cli/cmd_debug.cpp
105

What's wrong with just using chdir()? Is kyua going to run on windows?

This revision is now accepted and ready to land.Mon, Dec 29, 4:09 PM