Page MenuHomeFreeBSD
Differential D35830

cuse(3): Allow shared memory allocations up to and including 2 GBytes.
ClosedPublic
Actions

Authored by • hselasky on Jul 16 2022, 10:47 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Nov 6, 8:14 PM
Unknown Object (File)
Oct 27 2024, 4:20 AM
Unknown Object (File)
Oct 9 2024, 8:17 AM
Unknown Object (File)
Oct 2 2024, 6:38 AM
Unknown Object (File)
Oct 2 2024, 6:29 AM
Unknown Object (File)
Oct 2 2024, 6:28 AM
Unknown Object (File)
Oct 2 2024, 6:21 AM
Unknown Object (File)
Oct 2 2024, 4:33 AM
View All 98 Files
Subscribers
imp

Details

Reviewers
brooks
markj
• hselasky
Commits
rGa96a9c689ec8: cuse(3): Allow shared memory allocations up to, but excluding 2 GBytes.
rGd14b53ee31ca: cuse(3): Allow shared memory allocations up to, but excluding 2 GBytes.
Summary

Currently the cuse(3) mmap(2) offset is split into 128 banks of 16 Mbytes.
Allow cuse(3) to make allocations that span multiple banks at the expense
of any fragmentation issues that may arise. Typically mmap(2) buffers are
well below 16 Mbytes. This allows 8K video resolution to work using webcamd.

Some minor fixes are included aswell.

MFC after: 1 week
Sponsored by: NVIDIA Networking

Test Plan

V4L2 loopback with 8K resolution

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

• hselasky created this revision.Jul 16 2022, 10:47 AM
Herald added a subscriber: imp. · View Herald TranscriptJul 16 2022, 10:47 AM
• hselasky requested review of this revision.Jul 16 2022, 10:47 AM

Hopefull this works on CheriBSD aswell.

• hselasky added a reviewer: markj.Jul 18 2022, 1:41 PM

markj@ : Do you have time to have a quick look at this?

markj added inline comments.Jul 18 2022, 2:46 PM
lib/libcuse/cuse.3
128 ↗(On Diff #108229)

limited by what?

lib/libcuse/cuse_lib.c
208

Don't you need to reserve all units in the run, not just the first one? What if two threads concurrently attempt an allocation, one of 32MB and the other of 16MB.

284

Shouldn't this zero the buffer before unmapping? Since we're mapping with MAP_SHARED, any time a segment is recycled, I believe the old contents will be preserved.

• hselasky updated this revision to Diff 108279.Jul 18 2022, 3:05 PM
• hselasky marked an inline comment as done.

Update manual page as suggested by Mark.

• hselasky retitled this revision from cuse(3): Allow shared memory allocations up to and including 2 GBytes. to cuse(3): Allow shared memory allocations up, but excluding 2 GBytes..Jul 18 2022, 3:06 PM
• hselasky added inline comments.Jul 18 2022, 3:09 PM
lib/libcuse/cuse.3
128 ↗(On Diff #108229)

It is limited by the mmap offset being 32-bit also for 64-bit platforms due to 32-bit user-space programs. I'll get that in.

lib/libcuse/cuse_lib.c
166

I might want to change the argument to unsigned to support the 2GByte case.

208

If the size is set and you allocate 32MB it will skip two entries because one entry can only hold 16MB.
Meaning any concurrent threads will see that window a busy or occupied, due to the "check if there are m free units ahead".

284

I want that it's the applications responsibility to clear memory before free. Also, if the application crashes, this needs to be done in the kernel and not here.

• hselasky updated this revision to Diff 108281.Jul 18 2022, 3:16 PM
• hselasky retitled this revision from cuse(3): Allow shared memory allocations up, but excluding 2 GBytes. to cuse(3): Allow shared memory allocations up to and including 2 GBytes..

Make size argument for malloc unsigned. Really we want to use size_t, but because all allocations are limited to 32-bits it is kept 32-bit for now. Will need to introduce some symbol versioning to make this smooth.

markj added a comment.Jul 18 2022, 3:58 PM

I can't spot any obvious problems with this diff.

lib/libcuse/cuse_lib.c
177

The cast shouldn't be necessary.

208

Ok, I see now.

284

It's moot anyway, I see now that CUSE_IOCTL_FREE_MEMORY will deallocate the pager for the unit. So the next allocation will get zeroed pages no matter what.

• hselasky updated this revision to Diff 108313.Jul 19 2022, 6:35 AM

Remove not needed cast to unsigned long.

• hselasky marked 4 inline comments as done.Jul 19 2022, 6:36 AM
• hselasky marked an inline comment as done.

Thank you Mark for taking the time to review this.

• hselasky accepted this revision.Jul 20 2022, 8:32 AM
This revision is now accepted and ready to land.Jul 20 2022, 8:32 AM
Closed by commit rGd14b53ee31ca: cuse(3): Allow shared memory allocations up to, but excluding 2 GBytes. (authored by • hselasky). · Explain WhyJul 20 2022, 8:47 AM
This revision was automatically updated to reflect the committed changes.
• hselasky added a commit: rGd14b53ee31ca: cuse(3): Allow shared memory allocations up to, but excluding 2 GBytes..
• hselasky added a commit: rGa96a9c689ec8: cuse(3): Allow shared memory allocations up to, but excluding 2 GBytes..Jul 27 2022, 9:38 AM

Revision Contents
Changeset List

PathSize
lib/
libcuse/
91 lines
sys/
fs/
cuse/
111 lines
2 lines
6 lines

Diff 108359

View Options

lib/libcuse/cuse_lib.c

Loading...
View Options

sys/fs/cuse/cuse.c

Loading...
View Options

sys/fs/cuse/cuse_defs.h

Loading...
View Options

sys/fs/cuse/cuse_ioctl.h

Loading...