Add more documentation regarding unsafe AIO requests.

[jhb]

Summary

Add more documentation regarding unsafe AIO requests.

The asynchronous I/O changes made previously result in different
behavior out of the box. Previously all AIO requests failed with
ENOSYS / SIGSYS unless aio.ko was explicitly loaded. Now, some AIO
requests complete and others ("unsafe" requests) fail with EOPNOTSUPP.

Reword the introductory paragraph in aio(4) to add a general
description of AIO before describing the vfs.aio.enable_unsafe sysctl.

Remove the ENOSYS error description from aio_fsync(2), aio_read(2),
and aio_write(2) and replace it with a description of EOPNOTSUPP.

Remove the ENOSYS error description from aio_mlock(2).

Log a message to the system log the first time a process requests an
"unsafe" AIO request that fails with EOPNOTSUPP. This is modeled on
the log message used for processes using the legacy pty devices.

Test Plan

• Hack one of the aio tests to not check the unsafe sysctl and verify it triggers the log message in /var/log/messages.

[kib]

lib/libc/sys/aio_write.2

125 ↗

(On Diff #18211)

s/unsafe/aio-unsafe/

(and perhaps directly reference .Xr aio 4).

It is actually not the filedescriptor which is unsafe, but our aio implementation.

share/man/man4/aio.4

53 ↗

(On Diff #18211)

May be, add a term there, 'aio-unsafe' filedescriptor, and provide the definition.

sys/kern/vfs_aio.c

1696 ↗

(On Diff #18211)

Should this be atomic_cmpxchg_int() loop ? On somebody' bad day this would race and underflow.

[jhb]

lib/libc/sys/aio_write.2

125 ↗

(On Diff #18211)

Yes, the language is odd. I've reworded these to say that the specific operation is unsafe.

share/man/man4/aio.4

53 ↗

(On Diff #18211)

I've explicitly defined the operations that can block a daemon as unsafe which I think achieves this.

sys/kern/vfs_aio.c

1696 ↗

(On Diff #18211)

We don't use atomics in other places (the code I copied this from in particular). The user could fix it by writing to the sysctl at least.

I think if we had some sort of 'atomic_dec_nonneg' or something that returned the new value then I would be fine using that here (and fixing the other instance in the pty code).

[jhb]

• Define "unsafe" operations and treat those as unsafe rather than fd's.

[jhb]

Rebase.

[kib]

I looked at the unsafe stuff once more, and I think that it is too restrictive. My main point is that typical io operation on the VREG/VDIR file is almost always safe. The only exception I can think about is the NFS vnode on wedged server, and even this probably matter only for intr mounts.

Really aio-unsafe files are fifos/pipes, after sockets were fixed by your work.

sys/kern/vfs_aio.c

1696 ↗

(On Diff #18211)

We do not need a new atomic op there, some routine similar to pps_ratecheck() would be enough. I will code it up.

[kib]

sys/kern/vfs_aio.c

1696 ↗

(On Diff #18211)

See D7270.

[jhb]

In D7151#151193, @kib wrote:

I looked at the unsafe stuff once more, and I think that it is too restrictive. My main point is that typical io operation on the VREG/VDIR file is almost always safe. The only exception I can think about is the NFS vnode on wedged server, and even this probably matter only for intr mounts.

Really aio-unsafe files are fifos/pipes, after sockets were fixed by your work.

I erred on the side of caution in terms of only declaring things I know are safe. I think a patch to adjust the AIO code to not treat local filesystems as unsafe (assuming they really are) would be great, but also orthogonal to this change.

[jhb]

Rebase and use counted_warning().