Mark tests that cannot run in parallel as is_exclusive
Needs RevisionPublic
Actions

Authored by jmmv on Nov 22 2015, 8:30 PM.

Details

Reviewers

ngie
asomers

Group Reviewers

tests

Summary

Kyua 0.12 introduces support to run test cases in parallel.
Such support is currently disabled by default because 1) the code is
very new and 2) not all tests can be run in parallel safely. It is
up to the user to requrest this functionality.

Tests with side-effects, or tests that use unique system resources,
must be marked with is_exclusive=true to tell Kyua to run them on
their own. I have caught a couple of obvious cases that need this
property, but I'm sure there are more.

Kyua 0.11 chokes on unknown properties so this change can only be
committed once Kyua 0.12 is widely available in the binary package
repositories, Jenkins is updated, and Jenkins is confirmed to work
with the new release.

Diff Detail

Repository

rS FreeBSD src repository - subversion

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 4914
Build 4977: arc lint + arc unit

Event Timeline

jmmv updated this revision to Diff 10428.Nov 22 2015, 8:30 PM

jmmv retitled this revision from to Mark tests that cannot run in parallel as is_exclusive.

jmmv updated this object.

jmmv edited the test plan for this revision. (Show Details)

Herald added a subscriber: imp. · View Herald TranscriptNov 22 2015, 8:30 PM

jmmv added a reviewer: tests.Nov 22 2015, 8:30 PM

I think this is a good start.

This revision is now accepted and ready to land.Nov 22 2015, 10:43 PM

emaste added a subscriber: emaste.Nov 23 2015, 4:45 PM

Please add is_exclusive to the following files. They both use global resources

tests/etc/rc.d/routing_test.sh
tests/sys/netinet/fibs_test.sh

Also mark sys/geom/class as exclusive

This revision now requires review to proceed.Aug 24 2016, 10:48 AM

Also mark etc/rc.d/routing_test and sys/netinet/fibs_test as requested by asomers

Also, add is_exclusive to sys/kqueue/kqueue_test (or you could fix that test to use mkstemp)

jmmv mentioned this in rS304797: Make use of Kyua's work directories..Aug 25 2016, 10:28 AM

I've fixed sys/kqueue/kqueue_test for correctness reasons, but it needn't fixing from a parallelization point of view. The driver of those tests is a TAP test program, and Kyua cannot parallelize the execution of individual test cases within a TAP test program.

asomers accepted this revision.Aug 25 2016, 3:13 PM

asomers added a reviewer: asomers.

This revision is now accepted and ready to land.Aug 25 2016, 3:13 PM

Wouldn't anything that also sets allow_sysctl_side_effects be affected by this, e.g. tests/sys/kern/kern_descrip_test.c?

This revision now requires changes to proceed.Aug 25 2016, 5:03 PM

Sidenote: a lot of the non-ATF tests might be affected by this as well, e.g. tests/sys/mac .

ngie mentioned this in rS309469: MFC r304797,r305467,r305468,r305483:.Dec 3 2016, 2:55 AM

ngie mentioned this in rS313505: MFC r304797,r305467:.Feb 10 2017, 2:22 AM

asomers requested changes to this revision.Jul 30 2018, 9:37 PM

asomers added inline comments.

tests/etc/rc.d/Makefile
6	I don't think this should be necessary. What global resources do these tests access?
tests/sys/geom/class/Makefile.inc
7	This is obsolete. I fixed it the right way in r327666