Page MenuHomeFreeBSD

Use va_bytes >= va_size as a hint that a file is not sparse for vn_generic_copy_file_range()
AcceptedPublic

Authored by rmacklem on Mar 11 2024, 8:39 PM.
Tags
None
Referenced Files
F82296014: D44309.diff
Sat, Apr 27, 10:03 AM
F82235733: D44309.id135643.diff
Fri, Apr 26, 7:54 PM
Unknown Object (File)
Sun, Apr 14, 5:47 PM
Unknown Object (File)
Fri, Apr 5, 3:58 PM
Unknown Object (File)
Fri, Apr 5, 1:10 AM
Unknown Object (File)
Mar 18 2024, 5:40 AM
Unknown Object (File)
Mar 14 2024, 6:46 PM
Unknown Object (File)
Mar 14 2024, 5:39 PM
Subscribers

Details

Summary

vn_generic_copy_file_range() tries to maintain holes
is file ranges being copied, using SEEK_DATA/SEEK_HOLE
where possible,

Unfortunately SEEK_DATA/SEEK_HOLE operations can take
a long time under certain circumstances.
Although it is not currently possible to know if a file has
unallocated data regions, the case where va_byytes >= va_size
is a strong hint that there are no unallocated data regions.
This hint does not work well for file systems doing compression,
but since it is only a hint, it is still useful.

For the case of va_bytes >= va_size, avoid doing SEEK_DATA/SEEK_HOLE.

Test Plan

Tested for a variety of sparse and non-sparse files
being copied, both on local UFS and ZFS volumes,
as well as when these volumes are mounted via NFSv4.2.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

rmacklem retitled this revision from Use va_bytes >= va_size as ahint that a file is not sparse for vn_generic_copy_file_range() to Use va_bytes >= va_size as a hint that a file is not sparse for vn_generic_copy_file_range().Mar 11 2024, 8:43 PM
sys/kern/vfs_vnops.c
3310

I do not think this is true. For instance, for UFS, ext attr blocks are accounted in va_bytes, but not in va_size. So the file might be sparse still.

I suspect that the way forward is to add a 'va_holey' field to va_attr and calculate it in VOP_GETATTR().

Updated the comment as suggested by kib@.

va_holey may be the way to go for FreeBSD15, so long as
it is easy to calculate. Having a file system do something
akin to Seek_Hole in VOP_GETATTR() would defeat the purpose.

This patch will help for FreeBSD13/14 for at least UFS exports,
I think?

ZFS can be handled by setting vfs.zfs.dmu_offset_next_sync=0
as found by wollman@. See the email thread on freebsd-stable@
with subject "13-stable NFS server hang". I plan on documenting
this in the BUGS section of "man nfsd" unless someone has a better
suggestion? I also have a patch that adds a sysctl to limit the size of
a Copy, which can be used as a "last resort" for limiting the time a
NFSv4.2 Copy RPC takes.

sys/kern/vfs_vnops.c
3310

Does this updated comment look better?

va_holey may be the way to go for FreeBSD15, so long as
it is easy to calculate. Having a file system do something
akin to Seek_Hole in VOP_GETATTR() would defeat the purpose.

I specifically thought about va_holey for UFS as not needing SEEK_HOLE.
Basically for UFS, we should calculate number of ext blocks (the number of non-zero elements
in di_extb[]), then compare (di_blocks - ext blocks) * block size and di_size.
I.e. it is almost same as your test, but exclusing ext blocks from the comparision.

This patch will help for FreeBSD13/14 for at least UFS exports,
I think?

ZFS can be handled by setting vfs.zfs.dmu_offset_next_sync=0
as found by wollman@. See the email thread on freebsd-stable@
with subject "13-stable NFS server hang". I plan on documenting
this in the BUGS section of "man nfsd" unless someone has a better
suggestion? I also have a patch that adds a sysctl to limit the size of
a Copy, which can be used as a "last resort" for limiting the time a
NFSv4.2 Copy RPC takes.

sys/kern/vfs_vnops.c
3310

Yes.

This revision is now accepted and ready to land.Mar 12 2024, 2:42 PM