Context: the goal of bypass is to access the chosen counter more
directly, bypassing the FFclock counter synthesis mechanism, to reduce
timestamping latency. It is essential that the direct and FFclock based
counters correspond, that is have the same width and origin.
The main use case is for TSC-derived counters, as they are natively
FFclock compatible, and are directly accessible from userspace as well
as the kernel via rdtsc(), dramatically reducing access latency.
BYPASS
Currently the bypass variable sysctl_kern_ffclock_ffcounter_bypass is
defined (as static in kern_ffclock.c) but never used. This commit makes
it available as an extern in timeffc.c, the employs it as originally
intended in kern_tc.c :
- ffclock_read_counter : via rdtsc
- sysclock_getsnapshot : via rdtsc32 combined with th->th_offset_count to provide a consistent (FB,FF) snapshot.
Also in sysclock_getsnapshot, on the same theme of latency reduction,
the counter read is moved up as early as possible to optimize the raw
timestamp. The comment block is improved to describe the role and
benefit of the function and bypass.
The use of bypass is only supported for TSC-derived counters. The onus
is on the operator to set sysctl_kern_ffclock_ffcounter_bypass
appropriately. User applications can detect if bypass is active, and
choose to exploit it, or use the slower syscall access to the FFclock
counter instead.
COUNTER ORIGIN
Unlike FB clocks, the origin of the underlying counter in FFclocks plays
a key role. When the timecounter is changed, the FF counter synthesized
by FFclock is changed and its origin must be carefully set.
Changes in kern_tc.c:ffclock_change_tc :
- bug: the origin was not set coherently for the new counter. It is now set properly, and in such a way that the FB and FF counters are fully consistent. This requires the addition of new argument passing the new FBclock counter read.
- in the case of TSC counters, where bypass is possible, the origin setting requirements are more detailed (see comments in code). The new code supports a consistent underlying TSC counter which never changes origin between boots, irrespective of now many times the selected timecounter is changed, thereby always being in agreement with the hardware counter and rdtsc.
- some reorganisation to better structure the resetting of FFclock parameters for this tick.
- addition of verbosity to flag a change in the FFcounter, a rare and significant event.
- improved comment block.
kern_tc.c:tc_windup :
- addition of verbosity to flag a change in the selected counter, a rare and significant event.