Add a virtiofs file system based on FUSE to interface with the virtio fs device. The file system communicates to a FUSE server on the host through this device.
Details
Details
Diff Detail
Diff Detail
- Repository
- rG FreeBSD src repository
- Lint
Lint Passed - Unit
No Test Coverage - Build Status
Buildable 59022 Build 55909: arc lint + arc unit
Event Timeline
Comment Actions
I'm still a little shaky on the details of virtio-fs, but I can't find anything obviously wrong here.
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 5 | I think we're not supposed to use "All rights reserved" in new files. | |
Comment Actions
Thanks a lot for working on this!
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 419 | Can we use the "from" for the tag like p9fs does? It's a bit strange to have an unused arbitrary string as the "from", and I haven't managed to get root mount to work even though it seemingly is passing the options… | |
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 393 | The "strict access policy" is only breaking things for virtiofs, i.e. if you mount a home directory for a user, that user wouldn't be able to access it. | |
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 419 | ah, re: mountroot, virtiofs needs to be added to vfs_mountroot_wait_if_neccessary (ugh why don't we have proper registered flags for this)… And then, remount support is expected because it mounts as RO first. | |
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 419 | Actually that whole routine is bogus, as is waiting for the dev under the mountpoint. I've written it locally, but it was on a laptop that decided to die suddenly for reasons I've not been able to diagnose. | |
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 375 | You also have to *set* the size here with fiov_adjust like the local FUSE transport does when it copies data from userspace in fticket_aw_pull_uio right after fuse_body_audit. Took me a few hours to debug why getdirentries was completely busted (: It relies on the buffer size to stop iteration; without fiov_adjust the buffer size was always 4096 so it would iterate past the actual message size, hitting a wall of zeroes, interpreting that as an entry with a 0-length name and bailing with EINVAL. With that fixed, virtiofs is already looking a lot better than p9fs. Namely, I could install a package with pkg under a virtiofs root, while under p9fs I'm hitting rather strange errors. Reusing the FUSE protocol was a really clever idea! | |
Comment Actions
Thanks for the feedback! I will update the diff accordingly.
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 375 | Great catch! I think this also fixes a bug I had been trying to triage for write back mode when heavily loading the VM. I will update and fix. | |
| 419 | Sounds good, just to make sure I've gotten it right: Changing the VFS option from "tag" to "from" should bring virtiofs mount options in line w/ other file systems. | |
| 419 | Re: remount, I can add it to this diff if it helps with testing. I omitted it to keep this diff set as small as possible, because it is already pretty large. | |
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 124 | This just causes all buffers to leak. | |
Comment Actions
A couple more notes from testing:
There's some other place where FUSE layer can pick up zero-padded-to-power-of-two things, now with 1024 instead of 4096:
WARNING: FUSE protocol violation for server mounted at /: Returned an embedded NUL from FUSE_READLINK. This warning will not be repeated. Len 1024 0000 62 61 73 68 00 00 00 00 00 00 00 00 00 00 00 00 |bash............| 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| …
And trying to just find something in a src tree checkout, there are sporadic errors like
find: ./contrib/llvm-project/llvm/lib/DebugInfo/Symbolize: Bad file descriptor find: ./sys/dev/qat/qat_hw: Invalid argument find: ./sys/dev/qat/qat_api/common/stubs: Invalid argument // etc…
that seem completely random, on one invocation there are none, on another there's a couple…
| sys/fs/fuse/virtiofs_vfsops.c | ||
|---|---|---|
| 419 | yes, the "from" is the 'magic' one that's the source argument in the mount command that you have to pass anyway | |
| 472 | "virtiofs.virtio" (the last fs gets cut off) is kinda redundant, so this isn't necessary | |
Comment Actions
Thank you for the testing! Are you using writethrough or writeback mode for FUSE? Also, what command line arguments are you using for virtiofsd on the host side?
Comment Actions
Never adjusted any parameters on the client side. Adding --writeback to virtiofsd results in panic: FUSE: ticket still on answer delivery list.
I was using --inode-file-handles=prefer --allow-direct-io --allow-mmap mostly. But changing flags doesn't make the random errors go away.
(not surprising as write caching stuff shouldn't affect running find a bunch of times)