Page MenuHomeFreeBSD

pmc: better distinguish pmu-events allocation path
ClosedPublic

Authored by mhorne on Jun 24 2023, 7:20 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Nov 25, 5:58 AM
Unknown Object (File)
Sat, Nov 23, 11:24 PM
Unknown Object (File)
Sat, Nov 23, 4:02 AM
Unknown Object (File)
Thu, Nov 21, 1:11 PM
Unknown Object (File)
Tue, Nov 19, 10:17 PM
Unknown Object (File)
Tue, Nov 19, 5:56 AM
Unknown Object (File)
Thu, Nov 14, 6:35 PM
Unknown Object (File)
Wed, Nov 13, 6:14 PM
Subscribers

Details

Summary

Background:

The pm_ev field of struct pmc_op_pmcallocate and struct pmc
traditionally contains the index of the chosen event, corresponding to
the __PMC_EVENTS array in pmc_events.h. This is a static list of events,
maintained by FreeBSD.

In the usual case, libpmc translates the user supplied event name
(string) into the pm_ev index, which is passed as an argument to the
allocation syscall. On the kernel side, the allocation method for the
relevant hwpmc class translates the given index into the event code that
will be written to an event selection register.

In 2018, a new source of performance event definitions was introduced:
the pmu-events json files, which are maintained by the Linux kernel. The
result was better coverage for newer Intel processors with a reduced
maintenance burden for libpmc/hwpmc. Intel and AMD CPUs were
unconditionally switched to allocate events from pmu-events instead of
the traditional scheme (959826ca1bb0a, 81eb4dcf9e0d).

Under the pmu-events scheme, the pm_ev field contains an index
corresponding to the selected event from the pmu-events table, something
which the kernel has no knowledge of. The configuration for the
performance counting registers is instead passed via class-dependent
fields (struct pmc_md_op_pmcallocate).

In 2021 I changed the allocation logic so that it would attempt to
pull from the pmu-events table first, and fall-back to the traditional
method (dfb4fb41166bc3). Later, pmu-events support for arm64 and power8
CPUs was added (28dd6730a5d6 and b48a2770d48b).

The problem that remains is that the pm_ev field is overloaded, without
a definitive way to determine whether the event allocation came from the
pmu-events table or FreeBSD's statically-defined PMC events. This
resulted in a recent fix, 21f7397a61f7.

Change:

To disambiguate these two supported but separate use-cases, add a new
flag, PMC_F_EV_PMU, to be set as part of the allocation, indicating that
the event index came from pmu-events.

This is useful in two ways:

  1. On the kernel side, we can validate the syscall arguments better. Some classes support only the traditional event scheme (e.g. hwpmc_armv7), while others support only the pmu-events method (e.g. hwpmc_core for Intel). We can now check for this. The hwpmc_arm64 class supports both methods, so the new flag supersedes the existing MD flag, PM_MD_EVENT_RAW.
  2. The flag will be tracked in struct pmc for the duration of its lifetime, meaning it is communicated back to userspace. This allows libpmc to perform the reverse index-to-event-name translation without speculating about the meaning of the index value.

Adding the flag is a backwards-incompatible ABI change. We very recently
bumped the major version of the hwpmc module, so this breakage is
totally acceptable.

Test Plan

Testing performed for:

  • AMD (PMC_F_EV_PMU)
  • armv7 (!PMC_F_EV_PMU)
  • arm64 (both)

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Have you considered translating the older event definitions to JSON form, and dispensing with the __PMC_EVENTS macro?

Have you considered translating the older event definitions to JSON form, and dispensing with the __PMC_EVENTS macro?

This is a good question, I hadn't really considered it. It would take a big effort to perform the conversion all at once (a least 8 hwpmc classes, depending how you count); more time than I have before 14.0. To remove __PMC_EVENTS at a later time would require another major version bump. Even then, I can't say that it is an obvious win in terms of maintenance.

One thing I did consider was an alternate allocation op, i.e. PMC_OP_PMCALLOCATE2, and having the pmu-events path use this. However, the approach requires wider changes to the kernel just to get it operational, without much advantage. We would still need some kind of flag available to userspace for the pmclog.c case.

I would like to commit this soon, ideally before the stable/14 branch. @jkoshy any further thoughts?

I would like to commit this soon, ideally before the stable/14 branch. @jkoshy any further thoughts?

LGTM to unblock for now. The PMC APIs/programming model would need to be reworked to support systems with heterogeneous CPUs; we can look at this issue at that time.

This revision is now accepted and ready to land.Aug 20 2023, 5:53 PM