Page MenuHomeFreeBSD

Tighten up event scheduling
Needs ReviewPublic

Authored by jtl on Mar 24 2018, 2:42 PM.
Tags
None
Referenced Files
Unknown Object (File)
Jan 2 2024, 12:17 PM
Unknown Object (File)
Aug 31 2023, 9:45 AM
Unknown Object (File)
Apr 25 2023, 9:32 AM
Unknown Object (File)
Apr 7 2023, 4:21 PM
Subscribers

Details

Summary

Try to avoid IPIs to schedule events on other CPUs when not necessary. Also, try to avoid interrupts by looping if we are about to take an interrupt. Finally, get the current time much closer to actually setting the interrupt in order to get a more accurate time delta and reduce scheduling delays.

Test Plan

I ran this against a test which measured the delay encountered by direct timeouts. With this change (combined with two others), the delay from the deadline averaged 2.111 microseconds with a 2.505749 standard deviation.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 15773
Build 15785: arc lint + arc unit

Event Timeline

sys/kern/kern_clocksource.c
208

Is it possible that the function was called without work to do ? If this case has non-trivial number of occurrences, you might consider to calculate done under the lock and not dropping the lock if done == 0.

I'll need to look on it closer to recall all the details, but so far my main worry about your patches is that you are increase scope of ET_HW_LOCK(), which may be a problem for SMP system with non-per-CPU event timers, where that lock is global. I remember there were cases of 8-core systems using HPET by default, where this lock was pretty busy.

In D14827#311859, @mav wrote:

I'll need to look on it closer to recall all the details, but so far my main worry about your patches is that you are increase scope of ET_HW_LOCK(), which may be a problem for SMP system with non-per-CPU event timers, where that lock is global. I remember there were cases of 8-core systems using HPET by default, where this lock was pretty busy.

OK, if you think this will be a problem, I'll gladly accept suggestions that maintain correctness and improve performance.

sys/kern/kern_clocksource.c
208

I'll add a counter and check.

In D14827#311953, @jtl wrote:

OK, if you think this will be a problem, I'll gladly accept suggestions that maintain correctness and improve performance.

I don't have immediate solution. I agree that present code can be racy, but as I also see nexthard, nextstat and nextprof are accessed only by specific CPU and do not need a global lock you are protecting them with. Actually, as I see now, handleevents() is called only either from hardware interrupts or with interrupts disabled, so unless we support nested interrupts (which I am not sure), it should be safe without any additional locks. getnextcpuevent() on the other side may indeed benefit form additional locking, since it reads nextcall, which can be modified by other CPUs.

sys/kern/kern_clocksource.c
164

Where are you setting calloutready to false?