I think the tests are ok, though as before the errno checking doesn't seem very useful. I'm also not sure about this leak detection stuff, I would probably inline it into the timerfd.c tests if we're going to keep it. With ATF each test runs in its own process so fd leaks aren't really a problem, and many existing test cases are not careful to close fds.

A bit tangential, but: we found a while back that it was possible to trivially livelock a core by scheduling timeouts of 1ns using EVFILT_TIMER or setitimer(). Basically, it takes longer than the callout interval to execute the callout handler, so the callout handler ends up running in a loop.

As a mitigation, @kib introduced a mechanism to pause these callouts when the owning process receives SIGSTOP or SIGKILL. See itimer_proc_continue() and kqtimer_proc_continue(). Do we need a similar mechanism for timerfds?

In D55789#1276630, @markj wrote:

A bit tangential, but: we found a while back that it was possible to trivially livelock a core by scheduling timeouts of 1ns using EVFILT_TIMER or setitimer(). Basically, it takes longer than the callout interval to execute the callout handler, so the callout handler ends up running in a loop.

I should note that the current timerfd implementation fails to pass the timerfd__periodic_timer_performance test, which expects 400000000 individual nanosecond timeouts. I usually see 360000000 - 380000000 timeouts printed by the test case. I'm not sure if this performance issue is related to the core livelock in some way. The callout routine (timerfd_expire()) is fairly lightweight, specifying absolute expiration time with maximum precision (no C_PREL() or pr specified).

In D55789#1276630, @markj wrote:

A bit tangential, but: we found a while back that it was possible to trivially livelock a core by scheduling timeouts of 1ns using EVFILT_TIMER or setitimer(). Basically, it takes longer than the callout interval to execute the callout handler, so the callout handler ends up running in a loop.

As a mitigation, @kib introduced a mechanism to pause these callouts when the owning process receives SIGSTOP or SIGKILL. See itimer_proc_continue() and kqtimer_proc_continue(). Do we need a similar mechanism for timerfds?

Since these timers can be configured in periodic mode, the mitigation is required.

In D55789#1276669, @jfree wrote:

In D55789#1276630, @markj wrote:

A bit tangential, but: we found a while back that it was possible to trivially livelock a core by scheduling timeouts of 1ns using EVFILT_TIMER or setitimer(). Basically, it takes longer than the callout interval to execute the callout handler, so the callout handler ends up running in a loop.

I should note that the current timerfd implementation fails to pass the timerfd__periodic_timer_performance test, which expects 400000000 individual nanosecond timeouts. I usually see 360000000 - 380000000 timeouts printed by the test case. I'm not sure if this performance issue is related to the core livelock in some way. The callout routine (timerfd_expire()) is fairly lightweight, specifying absolute expiration time with maximum precision (no C_PREL() or pr specified).

Hmm, this sounds like another bug. Even if the callout handler is lightweight, I'm sure it plus the overhead of scheduling and running a callout costs more than 1ns. The livelock issue relates to the problem where it becomes impossible to schedule a thread on a looping core, because the softclock thread which runs the callout handler has high priority.

Hmm, this sounds like another bug. Even if the callout handler is lightweight, I'm sure it plus the overhead of scheduling and running a callout costs more than 1ns. The livelock issue relates to the problem where it becomes impossible to schedule a thread on a looping core, because the softclock thread which runs the callout handler has high priority.

So then the question becomes: what is the minimum amount of time, in the future, that a callout can be scheduled for without missing its activation time?

Say the next expiration is scheduled to happen in 1ns. Instead of actually scheduling that callout to active in 1ns, should I instead schedule the callout for 2ns in the future and when that activates increment the timerfd count +2?

I think this is especially tricky because it depends on system load, as you mentioned in D55790.

In D55789#1278377, @jfree wrote:

Hmm, this sounds like another bug. Even if the callout handler is lightweight, I'm sure it plus the overhead of scheduling and running a callout costs more than 1ns. The livelock issue relates to the problem where it becomes impossible to schedule a thread on a looping core, because the softclock thread which runs the callout handler has high priority.

So then the question becomes: what is the minimum amount of time, in the future, that a callout can be scheduled for without missing its activation time?

Say the next expiration is scheduled to happen in 1ns. Instead of actually scheduling that callout to active in 1ns, should I instead schedule the callout for 2ns in the future and when that activates increment the timerfd count +2?

I think this is especially tricky because it depends on system load, as you mentioned in D55790.

Our mitigation for this in other timer systems is to avoid making such estimates, and instead try to ensure that the system automatically stops the callout if the owning process is killed. See the P_SHOULDSTOP(p) || P_KILLED(p) checks in kern_time.c and kern_event.c.

In any case, I'm ok with this change as it is.

So then the question becomes: what is the minimum amount of time, in the future, that a callout can be scheduled for without missing its activation time?

Say the next expiration is scheduled to happen in 1ns. Instead of actually scheduling that callout to active in 1ns, should I instead schedule the callout for 2ns in the future and when that activates increment the timerfd count +2?

I think this is especially tricky because it depends on system load, as you mentioned in D55790.

Our mitigation for this in other timer systems is to avoid making such estimates, and instead try to ensure that the system automatically stops the callout if the owning process is killed. See the P_SHOULDSTOP(p) || P_KILLED(p) checks in kern_time.c and kern_event.c.

I was thinking more about the solution to the performance issue when I wrote that.

In any case, I'm ok with this change as it is.

I'll commit as is just to get this patch stack in.

In D55789#1276669, @jfree wrote:

In D55789#1276630, @markj wrote:

A bit tangential, but: we found a while back that it was possible to trivially livelock a core by scheduling timeouts of 1ns using EVFILT_TIMER or setitimer(). Basically, it takes longer than the callout interval to execute the callout handler, so the callout handler ends up running in a loop.

I should note that the current timerfd implementation fails to pass the timerfd__periodic_timer_performance test, which expects 400000000 individual nanosecond timeouts. I usually see 360000000 - 380000000 timeouts printed by the test case. I'm not sure if this performance issue is related to the core livelock in some way. The callout routine (timerfd_expire()) is fairly lightweight, specifying absolute expiration time with maximum precision (no C_PREL() or pr specified).

This is failing in CI as well: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/28085/testReport/sys.kern/timerfd/timerfd__periodic_timer_performance/

Could you file a bug to track this test failure, and add a atf_tc_expect_fail("https://bugs.freebsd.org/<PR>") to this testcase in the meantime?