Page MenuHomeFreeBSD

tests: skip gracefully when the tested binary's own package isn't installed
AcceptedPublic

Authored by imp on Wed, Jul 15, 5:39 PM.
Tags
None
Referenced Files
F163354725: D58262.id182043.diff
Wed, Jul 22, 11:01 AM
Unknown Object (File)
Tue, Jul 21, 9:01 PM
Unknown Object (File)
Sun, Jul 19, 8:04 PM
Unknown Object (File)
Sun, Jul 19, 2:47 AM
Unknown Object (File)
Sat, Jul 18, 3:51 AM
Subscribers

Details

Reviewers
olivier
ngie
Summary

pkgbase splits programs and their tests into separate packages (e.g.
unifdef/yacc/indent/file2c -> toolchain, jail -> jail, etc), while their
tests always land in the generic tests package. Installing just the
tests package therefore leaves these suites unable to find the binary
they exercise, and they fail confusingly instead of skipping.

Rather than force all packages to be installed for the tests package,
make the tests cope with the missing packages. Add require.progs (or,
for TAP/PLAIN-style suites with no atf_set hook: TEST_METADATA
required_programs) for the binary under test in: bectl, certctl,
ctfconvert, dhclient, file2c, indent, ipfw, jail, lastcomm, mixer,
newsyslog, nvmecontrol, pfctl, praudit, sa, syslogd, tunefs, unifdef,
yacc, ztest.

I skipped adding data for all the binaries in the base runtime
package for simplicity.

Assisted-by: Claude Code (Sonnet 5)
Sponsored by: Netflix

Test Plan

Testing is being done with VM images with fewer packages installed, though not all combos.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 74830
Build 71713: arc lint + arc unit

Event Timeline

imp requested review of this revision.Wed, Jul 15, 5:39 PM
imp added reviewers: olivier, ngie.
This revision is now accepted and ready to land.Wed, Jul 15, 8:23 PM

Side thought: it would be nice if these were actually broken down into subpackages, e.g., indent-tests, etc.

Side thought: it would be nice if these were actually broken down into subpackages, e.g., indent-tests, etc.

Not sure about this, but beyond the scope.

I also have one or two other changes to make this work I'll roll in before committing.

In D58262#1337132, @imp wrote:

Side thought: it would be nice if these were actually broken down into subpackages, e.g., indent-tests, etc.

Not sure about this, but beyond the scope.

I also have one or two other changes to make this work I'll roll in before committing.

Understandably so. I was thinking out loud because it's something that should be put down in our backlog, as an iterative next step towards breaking down how base is packaged.