makefs_zfs regression tests are all using the same zpool name, so are not compliant with parallel execution.
Details
Details
Without this patch:
VM# echo "parallelism=8" >> /etc/kyua/kyua.conf VM# cd /usr/tests/usr.sbin/makefs/ VM# kyua test makefs_zfs_tests makefs_zfs_tests:empty_dir -> failed: atf-check failed; see the output of the test for details [0.290s] makefs_zfs_tests:dataset_removal -> failed: atf-check failed; see the output of the test for details [0.322s] makefs_zfs_tests:hard_links -> failed: atf-check failed; see the output of the test for details [0.322s] makefs_zfs_tests:file_sizes -> failed: atf-check failed; see the output of the test for details [0.492s] makefs_zfs_tests:empty_fs -> passed [0.417s] makefs_zfs_tests:multi_dataset_1 -> failed: atf-check failed; see the output of the test for details [0.323s] makefs_zfs_tests:basic -> failed: atf-check failed; see the output of the test for details [0.783s] makefs_zfs_tests:autoexpand -> failed: atf-check failed; see the output of the test for details [0.838s] makefs_zfs_tests:multi_dataset_2 -> passed [0.408s] makefs_zfs_tests:multi_dataset_3 -> failed: atf-check failed; see the output of the test for details [0.285s] makefs_zfs_tests:multi_dataset_4 -> failed: atf-check failed; see the output of the test for details [0.222s] makefs_zfs_tests:multi_staging_2 -> failed: atf-check failed; see the output of the test for details [0.289s] makefs_zfs_tests:snapshot -> failed: atf-check failed; see the output of the test for details [0.195s] makefs_zfs_tests:multi_staging_1 -> passed [0.385s] makefs_zfs_tests:soft_links -> failed: atf-check failed; see the output of the test for details [0.252s] makefs_zfs_tests:root_props -> failed: atf-check failed; see the output of the test for details [0.806s] makefs_zfs_tests:long_file_name -> passed [1.371s] makefs_zfs_tests:indirect_dnode_array -> passed [3.035s] makefs_zfs_tests:reproducible -> passed [3.499s] Results file id is usr_tests_usr.sbin_makefs.20220824-230817-731224 Results saved to /root/.kyua/store/results.usr_tests_usr.sbin_makefs.20220824-230817-731224.db 6/19 passed (13 failed)
Diff Detail
Diff Detail
- Lint
Lint Skipped - Unit
Tests Skipped
Event Timeline
Comment Actions
makefs_zfs regression tests are all using the same zpool name
The poolname contains the PID, so I guess kyua is executing multiple tests in the same process context? How can I see the full test stdout/stderr when running multiple tests in parallel?
Comment Actions
Yes indeed, the PID are all diferrents so the poolname. The problem is elsewhere:
# kyua report --verbose
===> Execution context
Current directory: /usr/tests/usr.sbin/makefs
Environment variables:
BLOCKSIZE=K
EDITOR=vi
GROUP=wheel
HOME=/root
HOST=VM
HOSTTYPE=FreeBSD
LANG=C.UTF-8
LOGNAME=root
MACHTYPE=x86_64
MAIL=/var/mail/root
MM_CHARSET=UTF-8
OSTYPE=FreeBSD
PAGER=less
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
PWD=/usr/tests/usr.sbin/makefs
SHELL=/bin/csh
SHLVL=1
TERM=vt100
USER=root
VENDOR=amd
===> makefs_zfs_tests:autoexpand
Result: failed: atf-check failed; see the output of the test for details
Start time: 2022-08-25T00:19:46.948214Z
End time: 2022-08-25T00:19:47.796297Z
Duration: 0.848s
Metadata:
allowed_architectures is empty
allowed_platforms is empty
description is empty
has_cleanup = true
is_exclusive = false
required_configs is empty
required_disk_space = 0
required_files is empty
required_memory = 0
required_programs is empty
required_user is empty
timeout = 300
Standard output:
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/2/work/mnt ]
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/2/work/inputs ]
Executing command [ mkdir -m 0755 -p a/b/1 ]
Executing command [ ln -s a/b c ]
Executing command [ touch d ]
Executing command [ ln d e ]
Executing command [ touch .f ]
Executing command [ mkdir .g ]
Executing command [ dd if=/dev/zero of=i count=1000 bs=1 ]
Executing command [ touch klmn ]
Executing command [ touch opqr ]
Executing command [ touch stuv ]
Executing command [ install -m 0755 /dev/null wxyz ]
Executing command [ touch 0b00000001 ]
Executing command [ touch 0b00000010 ]
Executing command [ touch 0b00000011 ]
Executing command [ touch 0b00000100 ]
Executing command [ touch 0b00000101 ]
Executing command [ touch 0b00000110 ]
Executing command [ touch 0b00000111 ]
Executing command [ touch 0b00001000 ]
Executing command [ touch 0b00001001 ]
Executing command [ touch 0b00001010 ]
Executing command [ touch 0b00001011 ]
Executing command [ touch 0b00001100 ]
Executing command [ touch 0b00001101 ]
Executing command [ touch 0b00001110 ]
Executing command [ dd if=/dev/zero of=1.file bs=1 count=1 oseek=1 conv=sparse ]
Executing command [ dd if=/dev/zero of=512.file bs=1 count=1 oseek=512 conv=sparse ]
Executing command [ dd if=/dev/zero of=2048.file bs=1 count=1 oseek=2048 conv=sparse ]
Executing command [ dd if=/dev/zero of=10240.file bs=1 count=1 oseek=10240 conv=sparse ]
Executing command [ dd if=/dev/zero of=524288.file bs=1 count=1 oseek=524288 conv=sparse ]
Executing command [ makefs -t zfs -o nowarn=true -s 10737418240 -o mssize=134217728 -o rootpath=/ -o poolname=makefstest30970 /tmp/kyua.78oQpz/2/work/test.img /tmp/kyua.78oQpz/2/work/inputs ]
Executing command [ mdconfig -a -f /tmp/kyua.78oQpz/2/work/test.img ]
Executing command [ zpool import -R /tmp/kyua.78oQpz/2/work/mnt makefstest30970 ]
Standard error:
Fail: incorrect exit status: 1, expected: 0
stdout:
stderr:
cannot import 'makefstest30970': no such pool available
Files left in work directory after failure: inputs, md.output, mnt, test.img
cat: /tmp/kyua.78oQpz/2/work/poolname: No such file or directory
cannot open '': name must begin with a letter
===> makefs_zfs_tests:basic
Result: failed: atf-check failed; see the output of the test for details
Start time: 2022-08-25T00:19:46.949610Z
End time: 2022-08-25T00:19:47.786611Z
Duration: 0.837s
Metadata:
allowed_architectures is empty
allowed_platforms is empty
description is empty
has_cleanup = true
is_exclusive = false
required_configs is empty
required_disk_space = 0
required_files is empty
required_memory = 0
required_programs is empty
required_user is empty
timeout = 300
Standard output:
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/3/work/mnt ]
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/3/work/inputs ]
Executing command [ mkdir -m 0755 -p a/b/1 ]
Executing command [ ln -s a/b c ]
Executing command [ touch d ]
Executing command [ ln d e ]
Executing command [ touch .f ]
Executing command [ mkdir .g ]
Executing command [ dd if=/dev/zero of=i count=1000 bs=1 ]
Executing command [ touch klmn ]
Executing command [ touch opqr ]
Executing command [ touch stuv ]
Executing command [ install -m 0755 /dev/null wxyz ]
Executing command [ touch 0b00000001 ]
Executing command [ touch 0b00000010 ]
Executing command [ touch 0b00000011 ]
Executing command [ touch 0b00000100 ]
Executing command [ touch 0b00000101 ]
Executing command [ touch 0b00000110 ]
Executing command [ touch 0b00000111 ]
Executing command [ touch 0b00001000 ]
Executing command [ touch 0b00001001 ]
Executing command [ touch 0b00001010 ]
Executing command [ touch 0b00001011 ]
Executing command [ touch 0b00001100 ]
Executing command [ touch 0b00001101 ]
Executing command [ touch 0b00001110 ]
Executing command [ dd if=/dev/zero of=1.file bs=1 count=1 oseek=1 conv=sparse ]
Executing command [ dd if=/dev/zero of=512.file bs=1 count=1 oseek=512 conv=sparse ]
Executing command [ dd if=/dev/zero of=2048.file bs=1 count=1 oseek=2048 conv=sparse ]
Executing command [ dd if=/dev/zero of=10240.file bs=1 count=1 oseek=10240 conv=sparse ]
Executing command [ dd if=/dev/zero of=524288.file bs=1 count=1 oseek=524288 conv=sparse ]
Executing command [ makefs -t zfs -o nowarn=true -s 10g -o rootpath=/ -o poolname=makefstest30971 /tmp/kyua.78oQpz/3/work/test.img /tmp/kyua
.78oQpz/3/work/inputs ]
Executing command [ mdconfig -a -f /tmp/kyua.78oQpz/3/work/test.img ]
Executing command [ zpool import -R /tmp/kyua.78oQpz/3/work/mnt makefstest30971 ]
Standard error:
Fail: incorrect exit status: 1, expected: 0
stdout:
stderr:
cannot import 'makefstest30971': no such pool available
Files left in work directory after failure: inputs, md.output, mnt, test.img
cat: /tmp/kyua.78oQpz/3/work/poolname: No such file or directory
cannot open '': name must begin with a letter
===> makefs_zfs_tests:empty_dir
Result: failed: atf-check failed; see the output of the test for details
Start time: 2022-08-25T00:19:46.957465Z
End time: 2022-08-25T00:19:47.283094Z
Duration: 0.326s
Metadata:
allowed_architectures is empty
allowed_platforms is empty
description is empty
has_cleanup = true
is_exclusive = false
required_configs is empty
required_disk_space = 0
required_files is empty
required_memory = 0
required_programs is empty
required_user is empty
timeout = 300
Standard output:
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/5/work/mnt ]
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/5/work/inputs ]
Executing command [ makefs -t zfs -o nowarn=true -s 10g -o rootpath=/ -o poolname=makefstest30975 /tmp/kyua.78oQpz/5/work/test.img /tmp/kyua
.78oQpz/5/work/inputs ]
Executing command [ mdconfig -a -f /tmp/kyua.78oQpz/5/work/test.img ]
Executing command [ zpool import -R /tmp/kyua.78oQpz/5/work/mnt makefstest30975 ]
Standard error:
Fail: incorrect exit status: 1, expected: 0
stdout:
stderr:
cannot import 'makefstest30975': pool already exists
Files left in work directory after failure: inputs, md.output, mnt, test.img
cat: /tmp/kyua.78oQpz/5/work/poolname: No such file or directory
cannot open '': name must begin with a letter
===> makefs_zfs_tests:empty_fs
Result: failed: atf-check failed; see the output of the test for details
Start time: 2022-08-25T00:19:46.965942Z
End time: 2022-08-25T00:19:47.298847Z
Duration: 0.333s
Metadata:
allowed_architectures is empty
allowed_platforms is empty
description is empty
has_cleanup = true
is_exclusive = false
required_configs is empty
required_disk_space = 0
required_files is empty
required_memory = 0
required_programs is empty
required_user is empty
timeout = 300
Standard output:
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/6/work/mnt ]
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/6/work/inputs ]
Executing command [ makefs -t zfs -o nowarn=true -s 10g -o rootpath=/ -o poolname=makefstest30987 /tmp/kyua.78oQpz/6/work/test.img /tmp/kyua
.78oQpz/6/work/inputs ]
Executing command [ mdconfig -a -f /tmp/kyua.78oQpz/6/work/test.img ]
Executing command [ zpool import -R /tmp/kyua.78oQpz/6/work/mnt makefstest30987 ]
Standard error:
Fail: incorrect exit status: 1, expected: 0
stdout:
stderr:
cannot import 'makefstest30987': no such pool available
Files left in work directory after failure: inputs, md.output, mnt, test.img
cat: /tmp/kyua.78oQpz/6/work/poolname: No such file or directory
cannot open '': name must begin with a letter
===> makefs_zfs_tests:file_sizes
Result: failed: atf-check failed; see the output of the test for details
Start time: 2022-08-25T00:19:46.972400Z
End time: 2022-08-25T00:19:47.543610Z
Duration: 0.571s
Metadata:
allowed_architectures is empty
allowed_platforms is empty
description is empty
has_cleanup = true
is_exclusive = false
required_configs is empty
required_disk_space = 0
required_files is empty
required_memory = 0
required_programs is empty
required_user is empty
timeout = 300
Standard output:
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/7/work/mnt ]
Executing command [ mkdir -m 0777 -p /tmp/kyua.78oQpz/7/work/inputs ]
Executing command [ makefs -t zfs -o nowarn=true -s 10g -o rootpath=/ -o poolname=makefstest30998 /tmp/kyua.78oQpz/7/work/test.img /tmp/kyua
.78oQpz/7/work/inputs ]
Executing command [ mdconfig -a -f /tmp/kyua.78oQpz/7/work/test.img ]
Executing command [ zpool import -R /tmp/kyua.78oQpz/7/work/mnt makefstest30998 ]
Standard error:
Fail: incorrect exit status: 1, expected: 0
stdout:
stderr:
cannot import 'makefstest30998': no such pool available
Files left in work directory after failure: inputs, md.output, mnt, test.img
cat: /tmp/kyua.78oQpz/7/work/poolname: No such file or directory
cannot open '': name must begin with a letterComment Actions
I see now. The problem is that pools created by makefs always have the same GUID, so they can't be imported simultaneously. I added a parameter to makefs to specify a PRNG seed, and if I modify the tests to use it, they can run in parallel without problems. So, I don't think it'll be necessary to serialize the makefs_zfs tests.
Comment Actions
I tried several variants of this idea but was always able to trigger occasional collisions by running the ZFS tests in a loop overnight. So I ended up just committing a variant of this patch, together with some unrelated bug fixes.