I've implemented an improved encoding for large file descriptors that is efficient and only requires 16 additional bits from _flags2 instead of 31.
More importantly, when the file descriptor fits in a signed short, the 16 additional bits are all zero so the FILE struct data in memory is identical.
Given a 32-bit FD, compute FD ^ ((FD << 16) >> 16); the bottom word goes in_file, the top word goes in the new field.
I took inspiration from the AMD64 canonical virtual addresses scheme for this.
I've added the commit to the draft github pull request I mentioned earlier (URL below).
*https://github.com/freebsd/freebsd-src/pull/2005

I have reworked this change into a new branch off of CURRENT organised into three commits (with detailed, multi-line commit messages):

• 2aa94c66b4a4 stdio: Support 32-bit file descriptors in FILE
• 582805ad8d8d stdio: Remove fileno macros, add internal inline fileno get/set functions
• 23897d36c606 stdio: Thread safety fixes, sanity checks, initialisation fixes

I've reworked this (against releng/15.0) and put it in github here:
https://github.com/freebsd/freebsd-src/compare/releng/15.0...svenski123:freebsd-src:fd32compat

In D54355#1260829, @adrian wrote:

In D54355#1260823, @kris_tranception.com wrote:

I can see that int _flags2 was added to FILE ten years ago and only one bit has been defined leaving 31 unused bits, precisely what's needed to represent any non-negative file descriptor.
So I'm reworking this to not add a new field but rather continue using short _file for <32768 and the high 31 bits of _flags2 for descriptors >32767 && <0x7fffffff; the size of FILE will not need to change.

I'd hold off on that and have more of a chat about it first.

I can see that int _flags2 was added to FILE ten years ago and only one bit has been defined leaving 31 unused bits, precisely what's needed to represent any non-negative file descriptor.
So I'm reworking this to not add a new field but rather continue using short _file for <32768 and the high 31 bits of _flags2 for descriptors >32767 && <0x7fffffff; the size of FILE will not need to change.
Where the descriptor fits in a short, _flags2[31:1] can be set to 0x00007fff so that _flags2[31:1] is always non-zero indicating a FILE object constructed by new stdio vs. one constructed by old stdio; (mind you I'm not entirely convinced this is needed or useful).
I'll remove the fileno macro to prevent _flags2 getting compiled in and I'll add the versioned fopen and friends so old binaries get old short-only FILEs.
Not sure how to hook stdin/stdout/stderr construction and version it, looks like it's initialised at load time; likely doesn't matter unless apps freopen stdin/out/err and likely not even then.
This will be a bit more intrusive than the older patch but will hopefully satisfy backwards compatibility requirements so that it could get back ported for 15.1 and/or 14.4.

I have opened the following report in respect of the games/xpipeman and games/xrobots ports, attaching a patch patching the patch files in the ports such that they call fileno() instead accessing _file directly.

From the exp-run failure logs here...

Renaming _file is merely to identify ports that attempt access _file directly so that the consequences of widening _file on them can be reviewed and appropriate action taken.

In D54355#1256017, @jhb wrote:

Oh, this is the sort of thing I really hoped to avoid, was exposing all of FILE as a new ABI. I was careful in my previous versions to figure out which parts of FILE were actually used (not _fileno which doesn't matter, but all the _other_ fields that things like gnulib abuse) and then tried to make FILE mostly opaque, and added the wider _fileno in a slot that was safe to reuse that was near the front still. I think I also used some #ifdef's to ensure that only the fields in FILE that were part of the public ABI were exposed outside of libc. This doesn't do any of that.

@jhb Thank you for taking a look at this and apologies for my delay in responding as I've been on holiday the past week.

rebased to freebsd/main

• stdio: rename short _fileno to _fileno_short for legacy abi compatibility, add new int _fileno in struct __sFILE
• stdio: return error EBADF when fd < 0 in fdopen() and vdprintf()
• stdio: check file descriptor opened for writing in vdprintf()

rebased to freebsd/main

Remove inadvertent edit to comment (i.e. read_only_fileno should have remained file_only_file)