Page MenuHomeFreeBSD

linux: Avoid oversized native getdents64 batches
Needs ReviewPublic

Authored by hayzam_gmail.com on Fri, Oct 16, 2:22 AM.
Tags
None
Referenced Files
F176600591: D60760.diff
Tue, Oct 20, 3:30 AM
F176585323: D60760.id.diff
Tue, Oct 20, 12:56 AM
F176579570: D60760.diff
Tue, Oct 20, 12:00 AM
F176540712: D60760.id190009.diff
Mon, Oct 19, 4:36 PM
Unknown Object (File)
Sun, Oct 18, 11:08 PM
Unknown Object (File)
Sun, Oct 18, 9:14 AM
Unknown Object (File)
Sat, Oct 17, 6:38 PM
Unknown Object (File)
Sat, Oct 17, 12:49 PM
Subscribers

Details

Reviewers
markj
kevans
Summary

linux_getdents64() rounds its native read size up to DEV_BSIZE before
converting entries. For an unaligned caller buffer, the native batch
can contain more entries than fit in the Linux buffer. Conversion then
returns EINVAL even after copying complete entries, and the native read
has already advanced the directory position.

For counts of at least DEV_BSIZE, round the native batch down instead.
A 768-byte request, such as rustix's initial directory read, now reads
512 native bytes rather than 1024. The converted Linux dirent64 records
are no larger than their native counterparts, so the whole translated
batch fits. Returning a short batch is allowed, and consuming the whole
native batch avoids needing to rewind the position or change the NFS
client.

Keep the existing sizing for counts below DEV_BSIZE. This fixes the
oversized-batch case, not the remaining small-buffer, copyout-failure,
or legacy getdents/readdir problems.

Fixes: 110f2567cb51 ("linux_file.c: Fix handling of NFS getdents() emulation")

Test Plan

Reproduced the original EINVAL failure with the unmodified Zed 1.24.2 remote server on the stock kernel.
With the patch, Zed scans correctly on UFS, ZFS and NFSv4.2 mounts from FreeBSD and Linux servers.

Directory-read checks also passed over NFSv3, with and without READDIRPLUS,
covering buffer sweeps, shared fds, CAP_READ-only descriptors and 32-bit getdents64.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 78253
Build 75136: arc lint + arc unit

Event Timeline

hayzam_gmail.com edited the test plan for this revision. (Show Details)
sys/compat/linux/linux_file.c
564–571

Trying to wrap my head around this change, but what's wrong with the current assumption if we instead only consider this path an error if retval == 0? We might over-allocate for buf, but we'd presumably fit more into one call.

sys/compat/linux/linux_file.c
564–571

I believe it's the file position that gets in the way. kern_getdirentries() has already advanced past the whole native batch before this loop runs, so returning a short result here would skip the entries left in buf.

I tried that on UFS and ZFS: with a 768-byte buffer, enumeration completes without an error, but 112 of the 1002 entries are missing. This change returns all 1002. We'd need to preserve the remainder or arrange to commit the position after the last returned entry to handle partial batches correctly. Rounding down avoids that for counts >= DEV_BSIZE, though you're right that it can return fewer entries per call.

sys/compat/linux/linux_file.c
564–571

Ah, ok, so the cause here being that linux_getdirentries will loop on kern_getdirentries calls. VOP_READDIR implementations at places like [0] would normally avoid an EINVAL because we wrote something out, but this is a subsequent call to kern_getdirentries() -- we wrote something out in a previous one, but not in this one. linux_getdirentries can't reallly distinguish between this and other EINVAL cases, so we don't have much option beyond undersizing without plumbing something else through kern_getdirentries().

Does that sound like an accurate assessment?

[0] https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c#n1838

sys/compat/linux/linux_file.c
564–571

I added some tracing to understand this better. For the failing 768-byte request on UFS and ZFS, I see one successful kern_getdirentries() call producing 1000 native bytes, with no retry in linux_getdirentries(). EINVAL then comes from the conversion check: 720 Linux bytes have been copied, 48 remain in the caller’s buffer, and the next translated directory entry needs 96 bytes.

It looks like this reproducer is running out of space when converting into the Linux buffer, after the native read has completed. Could that be a different path from the retry case you had in mind?