Page MenuHomeFreeBSD

vmm: do not let the RTC's periodic interrupt drift
Needs ReviewPublic

Authored by wanpengqian_gmail.com on Fri, Oct 2, 8:19 AM.
Tags
None
Referenced Files
F174339668: D60233.id188375.diff
Fri, Oct 2, 12:56 PM
F174338778: D60233.diff
Fri, Oct 2, 12:44 PM
F174338191: D60233.diff
Fri, Oct 2, 12:37 PM
F174336782: D60233.diff
Fri, Oct 2, 12:19 PM
Subscribers

Details

Reviewers
jhb
markj
Group Reviewers
bhyve
Summary

The callout of the periodic interrupt was rearmed from its handler with
a timeout of one period from the time the handler ran. Each period was
longer than programmed by the latency of the callout, so the interrupt
came at a lower rate than the guest asked for. A guest that counts
these interrupts to keep its time runs slow.

Give the callout absolute deadlines one period apart, as vatpit does.
If a deadline has already passed when it is computed, for instance
after the VM was paused, start again from the current time.

Signed-off-by: Wanpeng Qian <wanpengqian@gmail.com>
Sponsored by: keelos.dev

Test Plan

main (f958aa7e7), FreeBSD 16.0-CURRENT guest with the RTC as its event timer (sysctl kern.eventtimer.timer=RTC), which makes the guest use the periodic interrupt. Interrupts on irq8 (vmstat -i) over 60 seconds of the guest's HPET time counter:

guestRTC rate programmedbeforeafter
kern.hz=100512 Hz458.2 per second (two runs: 458.16, 458.19)507.5 and 511.7 per second
kern.hz=10002048 Hz1492.0 per second1736 per second

The host of this test is itself a VM (FreeBSD main under QEMU/KVM, for want of a spare machine that runs main), so the callout latency is larger than on hardware. That is also why the rate is not exact after the change: when the handler runs more than one period late the deadline starts afresh (the same rule as in vatpit), and at 2048 Hz the guest does not acknowledge every interrupt before the next one is due.

On hardware (FreeBSD 14.5, Xeon Gold 5122, with other guests running): a Windows Server 2003 R2 guest with two vCPUs, which keeps its time by counting the RTC's periodic interrupt, lost 4.2 to 4.5 seconds in every 120 seconds (3.6 %). With the same change there, its offset from the host's clock stayed within 9 ms over 489 seconds.

rtcrate.sh
#!/bin/sh
# rtcrate.sh [SECONDS]: the rate of the RTC's periodic interrupt with the RTC as the event timer
T=${1:-60}
old=$(sysctl -n kern.eventtimer.timer)
sysctl kern.eventtimer.timer=RTC >/dev/null
sleep 3
c0=$(vmstat -i | awk '/atrtc/ {print $3}'); t0=$(date +%s.%N)
sleep $T
c1=$(vmstat -i | awk '/atrtc/ {print $3}'); t1=$(date +%s.%N)
sysctl kern.eventtimer.timer=$old >/dev/null
echo "$c0 $c1 $t0 $t1" | awk '{printf "RTC periodic interrupt: %d interrupts in %.1f s = %.2f per second\n", $2 - $1, $4 - $3, ($2 - $1) / ($4 - $3)}'

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 77608
Build 74491: arc lint + arc unit