fusefs: better test coverage for NFS-exported fuse file systems

[asomers]

Summary

nfsd is odd. It never calls VOP_OPEN or VOP_CLOSE. That causes it to
traverse code paths within fusefs that no userspace process ever will.
But nfsd is awkward to setup and teardown within ATF tests. So to get
coverage of these code paths, add a "-o simulate_nfs" hidden mount
option. When in effect, it will cause fusefs to ignore VOP_OPEN and
VOP_CLOSE, making all accesses approximately similar to what nfsd does.

Add one initial test case that uses this flag; more will follow.

Sponsored by: ConnectWise

[asomers]

@rmacklem do you think this does a decent job of approximating the peculiarities of nfsd?

[rmacklem]

I'm not sure I would have called the nfsd "odd". It is one of two
consumers of the VFS/VOP interface (I suppose you might call
the stacked file systems consumers as well?).

Being one of the two, it is definitely different than the local syscall
interface.

But, anyhow, it is definitely different than the local syscall interface
and not just not using VOP_OPEN/VOP_CLOSE. It uses VOP_LOOKUP,
but in a different way than the local syscalls, particularly w.r.t. name caching,
as another example.

If you find it useful for testing, I don't see a problem with doing it, but
I do have to wonder "If you want to test NFS, then why not just test NFS?".

[asomers]

In D48198#1099253, @rmacklem wrote:

I'm not sure I would have called the nfsd "odd". It is one of two
consumers of the VFS/VOP interface (I suppose you might call
the stacked file systems consumers as well?).

Being one of the two, it is definitely different than the local syscall
interface.

But, anyhow, it is definitely different than the local syscall interface
and not just not using VOP_OPEN/VOP_CLOSE. It uses VOP_LOOKUP,
but in a different way than the local syscalls, particularly w.r.t. name caching,
as another example.

I've noticed that nfsd will do VOP_LOOKUP calls for ".". Are there any other ways that it differs?

If you find it useful for testing, I don't see a problem with doing it, but
I do have to wonder "If you want to test NFS, then why not just test NFS?".

Because NFS exports are a global resource, which makes them difficult to setup and teardown during an ATF test. The /etc/exports file and the nfsd and rpcbind processes (and perhaps statd and mountd too) all need to have just a single user at a time. And the tests might clash with settings that are already in /etc/rc.conf and /etc/exports. Of course, if you know of a way to fix those problems so I can actually use nfsd in the test suite, that would be awesome.

[rmacklem]

In D48198#1099254, @asomers wrote:

In D48198#1099253, @rmacklem wrote:

I'm not sure I would have called the nfsd "odd". It is one of two
consumers of the VFS/VOP interface (I suppose you might call
the stacked file systems consumers as well?).

Being one of the two, it is definitely different than the local syscall
interface.

But, anyhow, it is definitely different than the local syscall interface
and not just not using VOP_OPEN/VOP_CLOSE. It uses VOP_LOOKUP,
but in a different way than the local syscalls, particularly w.r.t. name caching,
as another example.

I've noticed that nfsd will do VOP_LOOKUP calls for ".". Are there any other ways that it differs?

Not w.r.t. what it looks up, but in how it is set up.
Compare nfsvno_namei() with namei() in kern/vfs_lookup.c,
for instance.

Then there is the SMR stuff that the name cache code uses.
I don't understand what it is, but I frequently get panics
from it during development and basically just fiddle
with the code until the panic goes away.
(It assumes stuff set up by a process, which doesn't happen
for the kernel threads nfsd uses.)

If you find it useful for testing, I don't see a problem with doing it, but
I do have to wonder "If you want to test NFS, then why not just test NFS?".

Because NFS exports are a global resource, which makes them difficult to setup and teardown during an ATF test. The /etc/exports file and the nfsd and rpcbind processes (and perhaps statd and mountd too) all need to have just a single user at a time. And the tests might clash with settings that are already in /etc/rc.conf and /etc/exports. Of course, if you know of a way to fix those problems so I can actually use nfsd in the test suite, that would be awesome.

Exports, mountd and nfsd can be done in a vnet jail, if that would help?

[asomers]

In D48198#1099255, @rmacklem wrote:

Exports, mountd and nfsd can be done in a vnet jail, if that would help?

It certainly would help. Are there any good examples of code that creates a temporary vnet jail with nfs inside?

[rmacklem]

In D48198#1099283, @asomers wrote:

In D48198#1099255, @rmacklem wrote:

Exports, mountd and nfsd can be done in a vnet jail, if that would help?

It certainly would help. Are there any good examples of code that creates a temporary vnet jail with nfs inside?

I don't know if this helps:
https://people.freebsd.org/~rmacklem/nfsd-vnet-prison-setup.txt
I've always done it with a bsdinstall image. I don't know if that is temporaray enough?