Page MenuHomeFreeBSD

Fix `kyua list` for ATF_TESTS_KSH93 tests
Needs RevisionPublic

Authored by arichardson on Mar 4 2021, 6:05 PM.
Tags
None
Referenced Files
Unknown Object (File)
Dec 20 2023, 4:13 AM
Unknown Object (File)
Dec 23 2022, 8:17 PM

Details

Reviewers
asomers
lwhsu
Summary

Previously, we were executing these tests with
/usr/libexec/atf-sh -s/usr/local/bin/ksh93, but this causes kyua to
report a failed <testcase>:test_cases_list test instead of a skipped
test due to the missing ksh93. However, all of these scripts are actually
POSIX sh atf scripts that invoke ksh93 helpers, so we can execute them
using /bin/sh instead.

Before this change, running kyua test in /usr/tests without ksh93
installed reported 104 broken tests

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 37582
Build 34471: arc lint + arc unit

Event Timeline

asomers requested changes to this revision.Mar 4 2021, 6:57 PM

Your assertion that these are all POSIX sh scripts with ksh93 helpers is not correct. For example, hotspare/hotspare_test.sh sources hotspare/hotspare.kshlib, and that file makes use of ksh arrays. You therefore can't invoke hotspare_test.ksh with sh.

This revision now requires changes to proceed.Mar 4 2021, 6:57 PM

Your assertion that these are all POSIX sh scripts with ksh93 helpers is not correct. For example, hotspare/hotspare_test.sh sources hotspare/hotspare.kshlib, and that file makes use of ksh arrays. You therefore can't invoke hotspare_test.ksh with sh.

Thanks for pointing this out, I scanned 10 files and they all looked safe, so I assumed the .sh scripts were POSIX sh.

I guess the real fix would be for kyua to not bother running $test -l if required_programs is missing...

I guess the real fix would be for kyua to not bother running $test -l if required_programs is missing...

But if it doesn't do that, then it can't determine which test cases are being skipped.

Your assertion that these are all POSIX sh scripts with ksh93 helpers is not correct. For example, hotspare/hotspare_test.sh sources hotspare/hotspare.kshlib, and that file makes use of ksh arrays. You therefore can't invoke hotspare_test.ksh with sh.

Do you happen to know how many of the ATF scripts need to be executed as ksh93? Would it be feasible to change them to run the actual tests in a new ksh93 shell? Or maybe re-exec if $0 == sh?

I guess the real fix would be for kyua to not bother running $test -l if required_programs is missing...

But if it doesn't do that, then it can't determine which test cases are being skipped.

Yes good point, we probably do want to list the tests.

Your assertion that these are all POSIX sh scripts with ksh93 helpers is not correct. For example, hotspare/hotspare_test.sh sources hotspare/hotspare.kshlib, and that file makes use of ksh arrays. You therefore can't invoke hotspare_test.ksh with sh.

Do you happen to know how many of the ATF scripts need to be executed as ksh93? Would it be feasible to change them to run the actual tests in a new ksh93 shell? Or maybe re-exec if $0 == sh?

I don't know, and it isn't easy to tell. The actual tests _are_ executed in a new ksh93 shell. However, some of the included files must be included before executing the test. Switching the tests to atf-sh would require going through each one one at a time to remove ksh-isms.