@bz thanks, commit subject line and Reviewed-by's updated.

Great catch! I wonder what else that will help for the other (unconnected) drivers. Thanks!

Now in my staging tree as:

FYI, please generate diffs for reviews with full context e.g. git diff -U999999 or git show -U999999 so that Phabricator will display the additional context on demand. See https://wiki.freebsd.org/Phabricator for more info.

In D38459#950118, @jfree wrote:

This patch has been applied to src under commit af93fea710385b2b11f0cabd377e7ed6f3d97c34.

There doesn't seem to be a way to associate this review directly with that commit, so I'm just going to abandon it.

This patch has been applied to src under commit af93fea710385b2b11f0cabd377e7ed6f3d97c34.

In D38459#947334, @imp wrote:

Are there any tests?

I have this staged and plan to commit once I confirm universe builds.

I would like to get this patch committed before the 14.0 freeze in August. I am personally in favor of making these syscalls, but I'd like some notice if we're not going this route.

Add brief comments explaining each of the tfd_jumped macros.

In D38459#918469, @kib wrote:

In D38459#918467, @imp wrote:

I have code that automatically translates new system calls between host/target for bsd-user (though I'm not quite ready to commit it).
ioctl is possible, but has a lot more exceptional cases so is harder to do automatically. There's no automated annotation like there is for system calls.
The 'generic interfaces' require that I go and create annotations for each of the 'op codes' in that, because that doesn't exist in our current annotation.

So having a few extra system calls means they will be annotated and my job of getting it to work in bsd-user and others wanting to do automatic translation is easier.

For system calls you do need annotations, you cannot properly understand the purpose of any pointer passed to syscall otherwise. Is it in, out, in/out, or even just an abstract address like mmap/munmap/madvise arguments?

For system calls you do need annotations, you cannot properly understand the purpose of any pointer passed to syscall otherwise. Is it in, out, in/out, or even just an abstract address like mmap/munmap/madvise arguments?

For ioctls, the meaning of the command/arg is quite formalized, at least in BSDs. You are well aware that we encode both sizes and directions for copyin and copyout. After I took some time thinking how to implement what you described, I was surprised by the statement that syscalls are easier than ioctls.

That said, I am quite dislike extending the syscall semantical coverage by adding one-off operations. In the case of special fds, having all ops grouped together (under ioctl umbrella) localizes the interfaces and make them more comprehensive. This is, of course, my opinion, but all my experience supports the claim.

In D38459#918467, @imp wrote:

I have code that automatically translates new system calls between host/target for bsd-user (though I'm not quite ready to commit it).
ioctl is possible, but has a lot more exceptional cases so is harder to do automatically. There's no automated annotation like there is for system calls.
The 'generic interfaces' require that I go and create annotations for each of the 'op codes' in that, because that doesn't exist in our current annotation.

So having a few extra system calls means they will be annotated and my job of getting it to work in bsd-user and others wanting to do automatic translation is easier.

I have code that automatically translates new system calls between host/target for bsd-user (though I'm not quite ready to commit it).
ioctl is possible, but has a lot more exceptional cases so is harder to do automatically. There's no automated annotation like there is for system calls.
The 'generic interfaces' require that I go and create annotations for each of the 'op codes' in that, because that doesn't exist in our current annotation.

• tfd_jumped must be set to TFD_CANCELED, not TFD_JUMPED, for timerfd_settime to return ECANCELED.
• style(9) line-break fixes courtesy of @mckusick.

Er? Can you explain? What is confusing for ioctl, and what is intuitive for the syscall interface?

In D38459#914966, @jfree wrote:

Why? I do not like it very much. It is not as if all (or most) other individual kernel ops where performed by dedicated syscall entries, so I do not see a good reason to do it there. Ioctl perfectly fits the intent of providing file-specific requests.

The system call interface is standard and more intuitive for those that want to extend timerfd later.
The ioctl() implementation is confusing and messy for compat.

Er? Can you explain? What is confusing for ioctl, and what is intuitive for the syscall interface?

Why? I do not like it very much. It is not as if all (or most) other individual kernel ops where performed by dedicated syscall entries, so I do not see a good reason to do it there. Ioctl perfectly fits the intent of providing file-specific requests.

In D38459#914918, @jfree wrote:

• Support TFD_TIMER_CANCEL_ON_SET settime flag using timerfd_jumped().
• Convert ioctl() interface to native syscalls.

• Support TFD_TIMER_CANCEL_ON_SET settime flag using timerfd_jumped().
• Convert ioctl() interface to native syscalls.

I have contrary opinion about ioctl vs syscall.

In D38459#911504, @jfree wrote:

In D38459#903657, @val_packett.cool wrote:

• • and also…
• EVFILT_TIMER is currently subject to a system-wide and small-by-default kern.kq_calloutmax (kq_ncallouts) limit, which feels very unnerving: imagine an important daemon getting starved of timers by some random user app!! This code as-is is not, and I wouldn't like it to be, but then it would be kinda strange that EVFILT_TIMER would still be.
  • should we convert that to a per-{user,process,…} that both facilities would use? An rlimit sounds appropriate I think?
  • but do we need that limit at all? Maybe just abolish it from EVFILT_TIMER?

A per-proc limit sounds appropriate through rlimit. I'm not sure about abolishing the limit altogether, though. I am guessing it was implemented for a reason.

In D38459#903657, @val_packett.cool wrote:

Hey, couple random notes:

• re: "Developers that wish to support FreeBSD should avoid using timerfd" in the quarterly… :/
  • file descriptor handle based APIs are actually kinda better because composability / fd-passing / capability mode friendliness
    • FreeBSD invented procdesc(4) soo it's strange that we're not yet striving to turn everything into a file descriptor and Linux has us beat on this…
  • also there's no explicit clock selection in EVFILT_TIMER so when we finally add a suspend-aware monotonic clock it would only be possible to explicitly choose suspend-awareness-or-not with timerfd :)

Quick update: I'm nearly finished with my school work for the year, so I've had more time to work on this. I've nearly re-engineered the entire patch and I'm passing ~95% of the epoll-shim timerfd testing suite. I should have a new patch out in the next week (hopefully).

Hey, couple random notes:

In D38459#896885, @kib wrote:

What is the destiny of these patches? Why did you not committed them still?

What is the destiny of these patches? Why did you not committed them still?

In D38459#881824, @brooks wrote:

BTW, something that will need testing is to verify that the timerfd_gettime and timerfd_settime pseudo syscalls work in capability mode.

• Remove __FBSDID tag
• Move SYSINIT to avoid forward declaring timerfd_init()
• Correctly check for read blocking in FIONREADioctl case

Fix minor logic error in FIONREAD ioctl case

• Use SYSINIT() to initialize timerfd ino unit number
• Use atomic bitwise operations for file flags in FIONBIO ioctl
• Return 0 when FNONBLOCK is not set in FIONREAD ioctl

• Add generic ioctl() support for FIONBIO and FIONREAD

• M_TIMERFD malloc type is now static
• style(9) fixes
• TFD_GETTIME32 will only be defined on amd64 machines

BTW, something that will need testing is to verify that the timerfd_gettime and timerfd_settime pseudo syscalls work in capability mode.

• Use SV_CURPROC_FLAG() to determine if TFD_GETTIME32 compatibility instructions should be executed.

I fear I've found an issue with TFD_GETTIME32. The problem is that I believe struct itimerspec32 and struct itimerspecare the same size (but not the same layout) on non-x86 64-bit systems. As a result we need to use a different approach there. We still need to define TFD_GETTIME32 on amd64 due to the size (and thus command) difference there. I'll sketch the alternative code in the ioctl handler.

• Use _IOC_NEWTYPE() for COMPAT32 ioctls().

• Correct COMPAT32 pointer mistakes in struct timerfd_settime_args32
• tfdino_unr is now global static for unique ino generation

• struct itimerspec32 is now uint32_t in timerfd_settime_args32
• memset() itimerspec32s with zeros to avoid data leakage
• Remove DTYPE_TIMERFD checks for tfd fops.
• Various style(9) fixes

• Add tfd_ino field to struct timerfd for inode identification.

I'd personally name the functions user_timerfd_settime{,32}, but don't care that much.

Pass timerfd_settime_args into TFD_SETTIME ioctl() to minimize kernel entries.

Use timerfd_settime_user() to copyin timerfd_settime_args members.

The TFD_SETTIME ioctl() now returns old_time in an effort to avoid race conditions between threads using the same timerfd fd. This reduces the number of kernel traps from 3 to 2 in timerfd_settime().

Sorry, a cut-and-paste error made that make no sense. My initial worry was about TFD_GETTIME and TFD_SETTIME, but actually it's both. Consider the following scenario:

thread0       thread1
TFD_GETTIME
TFD_SETFLAGS
              TFD_GETTIME
              TFD_SETFLAGS
              TFD_SETTIME
TFD_SETTIME

I'm not sure how important it is to prevent this problem in practice, but multiple system calls seem to make problems more likely.

In D38460#879734, @jfree wrote:

In D38460#879425, @brooks wrote:

Is the race between TFD_GETTIME and TFD_SETTIME in timerfd_settime acceptable? I wonder if TFD_SETFLAGS should be _IOWR and always return the previous value.

Will execution continue in userspace before the kernel returns at timerfd.c:56? I'm not completely clear on how synchronicity works in the kernel for situations like this. I assumed the userspace thread would wait for the kernel to return before continuing execution. I don't see a reason for TFD_SETFLAGS to read the current flags considering Linux doesn't support similar functionality. Are you suggesting this to check for TFD_GETTIME completion?

Change file license from BSD-2-Clause-FreeBSD to BSD-2-Clause

• Add fo_stat implementation for timerfd
• ifdef _kernel TFD compat32 ioctl() macros

In D38460#879425, @brooks wrote:

Is the race between TFD_GETTIME and TFD_SETTIME in timerfd_settime acceptable? I wonder if TFD_SETFLAGS should be _IOWR and always return the previous value.

Is the race between TFD_GETTIME and TFD_SETTIME in timerfd_settime acceptable? I wonder if TFD_SETFLAGS should be _IOWR and always return the previous value.