fusefs: Fix intermittency in the BadServer.ShortWrite test case

[asomers]

Summary

We were using the m_quit bit for two similar but distinct uses:

• To instruct the server to quit
• To cope with the kernel forcibly unmounting the fs

Fix the intermittent test failure by adding a separate bit,
m_expect_unmount, to handle cases like the latter.

Reported by: Siva Mahadevan <[e-mail]>
MFC after: 1 week

[guest-svmhdvn]

Seems to fix the intermittency even with a very long runtime. I'm not the best person to review the changes, but it works well for me.

[guest-svmhdvn]

Sorry to bring this back, but I'm noticing intermittency again on latest 16-CURRENT HEAD. I can reproduce it locally on amd64, and I'm seeing it fail on aarch64 in jenkins with the same result: https://ci.freebsd.org/view/Test/job/FreeBSD-main-aarch64-test/1857/testReport/sys.fs.fusefs/bad_server/main/

I needed a large count (at least 500+ iterations on my local system) to reproduce it:

$ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:count=600 --param freebsd-src-regression-suite:tests='sys/fs/fusefs/bad_server:main'

Standard output:
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from BadServer
[ RUN      ] BadServer.UnknownUnique
[       OK ] BadServer.UnknownUnique (1 ms)
[ RUN      ] BadServer.ShortWrite
/home/siva/src/fbsdsrcgit/tests/sys/fs/fusefs/mockfs.cc:987: Failure
Value of: res >= static_cast<ssize_t>(sizeof(in.header)) || m_quit
  Actual: false
Expected: true

[  FAILED  ] BadServer.ShortWrite (0 ms)
[ RUN      ] BadServer.ErrorWithPayload
[       OK ] BadServer.ErrorWithPayload (0 ms)
[----------] 3 tests from BadServer (2 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (2 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] BadServer.ShortWrite

 1 FAILED TEST
===> Failed tests
sys/fs/fusefs/bad_server:main  ->  failed: Returned non-success exit status 1  [0.006s]

@asomers could you try to reproduce?

[asomers]

In D53357#1241293, @guest-svmhdvn wrote:

Sorry to bring this back, but I'm noticing intermittency again on latest 16-CURRENT HEAD. I can reproduce it locally on amd64, and I'm seeing it fail on aarch64 in jenkins with the same result: https://ci.freebsd.org/view/Test/job/FreeBSD-main-aarch64-test/1857/testReport/sys.fs.fusefs/bad_server/main/

I needed a large count (at least 500+ iterations on my local system) to reproduce it:

$ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:count=600 --param freebsd-src-regression-suite:tests='sys/fs/fusefs/bad_server:main'

Standard output:
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from BadServer
[ RUN      ] BadServer.UnknownUnique
[       OK ] BadServer.UnknownUnique (1 ms)
[ RUN      ] BadServer.ShortWrite
/home/siva/src/fbsdsrcgit/tests/sys/fs/fusefs/mockfs.cc:987: Failure
Value of: res >= static_cast<ssize_t>(sizeof(in.header)) || m_quit
  Actual: false
Expected: true

[  FAILED  ] BadServer.ShortWrite (0 ms)
[ RUN      ] BadServer.ErrorWithPayload
[       OK ] BadServer.ErrorWithPayload (0 ms)
[----------] 3 tests from BadServer (2 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (2 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] BadServer.ShortWrite

 1 FAILED TEST
===> Failed tests
sys/fs/fusefs/bad_server:main  ->  failed: Returned non-success exit status 1  [0.006s]

@asomers could you try to reproduce?

Good catch. I can reproduce too, but it takes a few thousand tries. See https://reviews.freebsd.org/D54331 for a fix.