Add unit test for ggate flush

[jo_bruelltuete.com]

Summary

Add a test case for flush in ggate.

And while here fix the problem it uncovers.

[jo_bruelltuete.com]

sbin/ggate/ggated/ggated.c

753

fsync() doesn't actually result in any flushes...
but i suppose we want both here? any opinions?

tests/sys/geom/class/gate/ggate_test.sh

30

how to trigger a bio_flush from user mode?
zpool works.
none of the fsync or dd with conv=fsync do, however.

39

cleanup is not great...

the cleanup() functions here are never called when i try to run this script.

[jo_bruelltuete.com]

tests/sys/geom/class/gate/ggate_test.sh

223

no, bad idea... this breaks the tests (they no longer fail on failure)

[jo_bruelltuete.com]

tests/sys/geom/class/gate/ggate_test.sh

37

this way around the test will only pass if numflushes has a number in it.

186

fill the whole device

232

64m is minimum pool size.