Page MenuHomeFreeBSD

lorder: skip tests if cc not available
Needs RevisionPublic

Authored by olivier on Mar 28 2024, 5:13 PM.
Tags
None
Referenced Files
F100222275: D44548.diff
Tue, Oct 15, 1:49 AM
Unknown Object (File)
Fri, Oct 4, 8:52 PM
Unknown Object (File)
Wed, Oct 2, 4:53 AM
Unknown Object (File)
Wed, Sep 18, 4:08 AM
Unknown Object (File)
Sep 9 2024, 7:29 AM
Unknown Object (File)
Jul 29 2024, 10:22 AM
Unknown Object (File)
Jul 23 2024, 9:47 AM
Unknown Object (File)
Jun 30 2024, 10:32 AM
Subscribers

Details

Reviewers
des
ngie
Group Reviewers
tests
Summary

On system without compilere installed the tests are failing:

 $ kyua test usr.bin/lorder/
usr.bin/lorder/lorder_test:noargs  ->  passed  [0.019s]
usr.bin/lorder/lorder_test:dashdash  ->  failed: atf-check failed; see the output of the test for details  [0.029s]
usr.bin/lorder/lorder_test:invalid  ->  passed  [0.029s]
usr.bin/lorder/lorder_test:nonexistent  ->  passed  [0.028s]
usr.bin/lorder/lorder_test:onearg  ->  failed: atf-check failed; see the output of the test for details  [0.028s]
usr.bin/lorder/lorder_test:objects  ->  failed: atf-check failed; see the output of the test for details  [0.030s]
usr.bin/lorder/lorder_test:archives  ->  failed: atf-check failed; see the output of the test for details  [0.045s]

3/7 passed (4 failed)

Add a check for cc on tests using it.

Test Plan

Skip if cc is missing:

# mv /usr/bin/cc /usr/bin/cc.bak
# kyua test usr.bin/lorder/
usr.bin/lorder/lorder_test:archives  ->  skipped: The required program cc could not be found in the PATH  [0.009s]
usr.bin/lorder/lorder_test:dashdash  ->  skipped: The required program cc could not be found in the PATH  [0.010s]
usr.bin/lorder/lorder_test:objects  ->  skipped: The required program cc could not be found in the PATH  [0.010s]
usr.bin/lorder/lorder_test:onearg  ->  skipped: The required program cc could not be found in the PATH  [0.010s]
usr.bin/lorder/lorder_test:noargs  ->  passed  [0.014s]
usr.bin/lorder/lorder_test:invalid  ->  passed  [0.020s]
usr.bin/lorder/lorder_test:nonexistent  ->  passed  [0.019s]

7/7 passed (0 failed)

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

olivier created this revision.

Why are you testing lorder, which is part of the toolchain, on a system without a toolchain?

In D44548#1015876, @des wrote:

Why are you testing lorder, which is part of the toolchain, on a system without a toolchain?

I’m just running the regression tests from a FreeBSD with a make.conf full of WITHOUT_
And since the introduction of those tests, I’m noticing those new tests are failing.
So one idea is too declare the requiere_prog to avoid that.

Now that I think of it, this isn't enough, because lorder itself uses nm, which is also conditional on MK_TOOLCHAIN. Also, the canonical way to do this is to add atf_set require.progs to the test case head, which allows Kyua to skip the test entirely, instead of trying to run it and then ignoring the failure.

In D44548#1016031, @des wrote:

I think D44558 is a better approach.

Yes: Thanks

Update test to latest sync, and add nm in requiere list too.

ngie requested changes to this revision.EditedWed, Oct 2, 2:23 AM
ngie added a subscriber: ngie.

Specifying this in the Makefile is significantly less code than doing it in the atf tests:

TEST_METADATA.lorder_test+=  required_programs="ar cc lorder nm"

This metadata ends up in the Kyuafile.

This revision now requires changes to proceed.Wed, Oct 2, 2:23 AM

Specifying this in the Makefile is significantly less code than doing it in the atf tests:

TEST_METADATA.lorder_test+=  required_programs="ar cc lorder nm"

This metadata ends up in the Kyuafile.

I didn’t know we could do that !
Thanks! I will fix it.

Specifying this in the Makefile is significantly less code than doing it in the atf tests:

TEST_METADATA.lorder_test+=  required_programs="ar cc lorder nm"

This metadata ends up in the Kyuafile.

Yes, it's less code, but I happen to know that lorder _and these tests_ are also used by downstream projects, and separating the requirements from the tests themselves will cause difficulties for those downstream consumers. Olivier's patch is fine as it is, although I'm not sure it is necessary to require lorder in a test meant to exercise lorder since if lorder is not installed, then neither is the test.

usr.bin/lorder/tests/lorder_test.sh
12

This is redundant since the test is only installed if lorder is installed.