Add a wrapping script to use ATF to run tests written with googletest
one by one. This helps locating and tracking the failing case in CI easier.
Before:
root@:/usr/tests/sys/capsicum # kyua test bindat_connectat:bindat_connectat_1 -> passed [0.005s] bindat_connectat:bindat_connectat_2 -> passed [0.004s] bindat_connectat:bindat_connectat_3 -> passed [0.004s] capsicum-test:main -> passed [37.380s] ioctls_test:cap_ioctls__listen_copy -> passed [0.005s] Results file id is usr_tests_sys_capsicum.20200730-195720-761226 Results saved to /root/.kyua/store/results.usr_tests_sys_capsicum.20200730-195720-761226.db 5/5 passed (0 failed)
After:
lwhsu@x1c:/usr/tests/sys/capsicum > kyua test bindat_connectat:bindat_connectat_1 -> passed [0.005s] bindat_connectat:bindat_connectat_2 -> passed [0.005s] bindat_connectat:bindat_connectat_3 -> passed [0.005s] functional:CapMode__AllowedMmapSyscallsForked -> passed [0.116s] functional:CapabilityPair__sendfile -> passed [0.410s] functional:Capability__BasicInterceptionForked -> passed [0.347s] functional:Capability__CapEnterForked -> passed [0.368s] functional:Capability__CapNewForked -> passed [0.499s] functional:Capability__DirOperationsForked -> passed [0.122s] functional:Capability__ExtendedAttributesForked -> passed [0.328s] functional:Capability__FileInSyncForked -> passed [0.365s] functional:Capability__InheritanceForked -> passed [0.366s] functional:Capability__MmapForked -> passed [0.348s] functional:Capability__NoBypassDAC -> passed [0.322s] functional:Capability__OpenAtDirectoryTraversalForked -> passed [0.325s] functional:Capability__OperationsForked -> passed [0.296s] functional:Capability__PipeUnseekable -> passed [0.284s] functional:Capability__SocketTransferForked -> passed [4.297s] functional:Capability__SyscallAt -> passed [0.090s] functional:Capmode__Abort -> passed [0.099s] functional:Capmode__AllowedAtSyscalls -> passed [0.098s] functional:Capmode__AllowedAtSyscallsCwd -> passed [0.124s] functional:Capmode__AllowedIdentifierSyscallsForked -> passed [0.121s] functional:Capmode__AllowedPipeSyscallsForked -> passed [0.393s] functional:Capmode__AllowedProfilSyscallForked -> passed [0.418s] functional:Capmode__AllowedResourceSyscallsForked -> passed [0.373s] functional:Capmode__AllowedSchedSyscallsForked -> passed [0.322s] functional:Capmode__AllowedTimerSyscallsForked -> passed [0.133s] functional:Capmode__NewThreadForked -> passed [3.399s] functional:Capmode__SelfKillForked -> passed [0.301s] functional:Fcntl__BasicForked -> passed [0.349s] functional:Fcntl__Commands -> passed [0.373s] functional:Fcntl__FLSubRights -> passed [0.355s] functional:Fcntl__OWNSubRights -> passed [0.355s] functional:Fcntl__PreserveSubRights -> passed [0.338s] functional:Fcntl__SubRightNormalFD -> passed [0.319s] functional:Fcntl__WriteLock -> passed [0.317s] functional:ForkedExecve_BasicFexecve___ -> passed [0.328s] functional:ForkedExecve_FailWithoutCap___ -> passed [0.354s] functional:ForkedExecve_InCapMode___ -> passed [0.333s] functional:ForkedExecve_SucceedWithCap___ -> passed [0.368s] functional:ForkedFexecveWithScript_CapModeScriptFail___ -> passed [0.347s] functional:ForkedFexecve_ExecutePermissionCheck___ -> passed [0.344s] functional:ForkedFexecve_ExecveFailure___ -> passed [0.328s] functional:ForkedFexecve_SetuidIgnored___ -> passed [0.331s] functional:ForkedOpenatTest_InCapabilityMode___ -> passed [0.365s] functional:ForkedOpenatTest_WithFlagInCapabilityMode___ -> passed [0.331s] functional:ForkedWithFiles_AllowedFileSyscalls___ -> passed [0.329s] functional:ForkedWithFiles_AllowedMiscSyscalls___ -> passed [0.376s] functional:ForkedWithFiles_AllowedSocketSyscalls___ -> passed [0.248s] functional:ForkedWithFiles_DisallowedFileSyscalls___ -> passed [0.299s] functional:ForkedWithFiles_DisallowedSocketSyscalls___ -> passed [0.209s] functional:Ioctl__Basic -> passed [0.356s] functional:Ioctl__PreserveSubRights -> passed [0.356s] functional:Ioctl__SubRightNormalFD -> passed [0.150s] functional:Ioctl__SubRights -> passed [0.103s] functional:Ioctl__TooManySubRights -> passed [0.115s] functional:OpenatTest__WithCapability -> passed [0.114s] functional:OpenatTest__WithFlag -> passed [0.108s] functional:Openat__RelativeForked -> passed [0.113s] functional:Pdfork__Bagpuss -> passed [6.539s] functional:Pdfork__BagpussDaemon -> passed [6.521s] functional:Pdfork__DaemonUnrestrictedForked -> passed [0.138s] functional:Pdfork__FromThread -> passed [2.238s] functional:Pdfork__InvalidFlag -> passed [0.116s] functional:Pdfork__MissingRights -> passed [0.112s] functional:Pdfork__NonProcessDescriptor -> passed [0.113s] functional:Pdfork__OtherUserForked -> passed [0.139s] functional:Pdfork__PdkillForked -> passed [1.441s] functional:Pdfork__PdkillOtherSignal -> passed [1.255s] functional:Pdfork__PdkillSignalForked -> passed [1.209s] functional:Pdfork__Simple -> passed [2.287s] functional:Pdfork__TimeCheck -> passed [0.117s] functional:Pdfork__UseDescriptor -> passed [0.125s] functional:PipePdforkDaemon__Close -> passed [0.234s] functional:PipePdforkDaemon__NoPDSigchld -> passed [0.415s] functional:PipePdforkDaemon__Pdkill -> passed [0.546s] functional:PipePdfork__ChildExit -> passed [0.467s] functional:PipePdfork__Close -> passed [0.483s] functional:PipePdfork__CloseLast -> passed [0.571s] functional:PipePdfork__ModeBits -> passed [0.493s] functional:PipePdfork__MultipleRetrieveExitStatus -> passed [0.519s] functional:PipePdfork__NoSigchld -> passed [0.409s] functional:PipePdfork__PassProcessDescriptor -> passed [3.463s] functional:PipePdfork__Pdkill -> passed [0.280s] functional:PipePdfork__Poll -> passed [0.411s] functional:PipePdfork__PollMultiple -> passed [1.421s] functional:PipePdfork__WaitPdThenPid -> passed [0.123s] functional:PipePdfork__WaitPidThenPd -> passed [0.116s] functional:PipePdfork__WildcardWait -> passed [0.123s] functional:Poll__LotsOFileDescriptorsForked -> passed [0.133s] functional:PosixMqueue__CapModeForked -> passed [0.427s] functional:RenameAt__AbsDesignationSame -> passed [0.410s] functional:Rename__AbsDesignationSame -> passed [0.411s] functional:Select__LotsOFileDescriptorsForked -> passed [0.288s] functional:Socket__TCP -> passed [1.439s] functional:Socket__UDP -> passed [0.105s] functional:Socket__UnixDomain -> passed [1.124s] ioctls_test:cap_ioctls__listen_copy -> passed [0.009s] Results file id is usr_tests_sys_capsicum.20200730-195445-302259 Results saved to /home/lwhsu/.kyua/store/results.usr_tests_sys_capsicum.20200730-195445-302259.db 99/99 passed (0 failed)