HomeFreeBSD

sched_ule: Use explicit atomic accesses for tdq fields

Description

sched_ule: Use explicit atomic accesses for tdq fields

Different fields in the tdq have different synchronization protocols.
Some are constant, some are accessed only while holding the tdq lock,
some are modified with the lock held but accessed without the lock, some
are accessed only on the tdq's CPU, and some are not synchronized by the
lock at all.

Convert ULE to stop using volatile and instead use atomic_load_* and
atomic_store_* to provide the desired semantics for lockless accesses.
This makes the intent of the code more explicit, gives more freedom to
the compiler when accesses do not need to be qualified, and lets KCSAN
intercept unlocked accesses.

Thus:

  • Introduce macros to provide unlocked accessors for certain fields.
  • Use atomic_load/store for all accesses of tdq_cpu_idle, which is not synchronized by the mutex.
  • Use atomic_load/store for accesses of the switch count, which is updated by sched_clock().
  • Add some comments to fields of struct tdq describing how accesses are synchronized.

No functional change intended.

Reviewed by: mav, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35737

Details

Provenance
markjAuthored on Jul 14 2022, 2:43 PM
Reviewer
mav
Differential Revision
D35737: ule: Use explicit atomic accesses for tdq fields
Parents
rG03f868b163ad: x86: Add a required store-load barrier in cpu_idle()
Branches
Unknown
Tags
Unknown