Page MenuHomeFreeBSD

hwpmc: Add support for AMD-only IRPERF in hwpmc PERF class
Needs ReviewPublic

Authored by anascime_amd.com on Thu, Sep 10, 2:36 AM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Sep 24, 2:34 PM
Unknown Object (File)
Thu, Sep 24, 2:33 PM
Unknown Object (File)
Wed, Sep 23, 11:55 PM
Unknown Object (File)
Wed, Sep 23, 11:53 PM
Unknown Object (File)
Tue, Sep 22, 9:32 PM
Unknown Object (File)
Tue, Sep 22, 1:49 PM
Unknown Object (File)
Mon, Sep 21, 5:16 AM
Unknown Object (File)
Sun, Sep 20, 5:58 AM
Subscribers

Details

Summary

The IRPERF MSR counter (0xc00000e9) exposes an instructions-retired count
when read. Its support is advertised through CPUID leaf 0x80000008 EBX bit 1.
If this bit is set, the CPU supports the IRPERF MSR. For it to be used
by the system, bit 30 of the HWCR MSR (0xC0010015) needs to be enabled.

This patch checks for availability of the feature and enables the bit. If
the feature is available, the hwpmc PERF class detects its presence and
registers a third PERF MSR counter that allows hwpmc users to measure
instructions retired using IRPERF. This is AMD-only, as Intel does not
support IRPERF.

Due to erratum #1054 affecting AMD Zen 1 before model 30h, IRPERF is only
enabled for AMD Zen 1 model 30h onwards and later microarchitectures. This
commit also implements a validation step to make sure the IRPERF counter
is actually incrementing; otherwise, the feature is not enabled. This
guarantees it is working as expected before being made available to the
system and users.

To support this, the perf_pmcdesc[] structure changed from being
statically initialized with 2 entries (MPERF/APERF) to being populated
dynamically. If both MPERF/APERF and IRPERF are supported, the structure
will contain three entries: MPERF, APERF, and IRPERF. Each entry now also
carries its own MSR address and PMC event, so perf_read_pmc() reads the
MSR to use from the matching perf_pmcdesc[] entry at runtime instead of
switching over a fixed set of row indices.

Definitions for the features available on the HWCR MSR did not previously
exist. This commit adds macros for those features, including
HWCR_IRPERFEN. The MSR definition for MSR_IRPERF was also missing and has
now been added.

This commit also updates the manual page accordingly.

Signed-off-by: Anderson Nascimento <anascime@amd.com>
Sponsored by: AMD

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 76945
Build 73828: arc lint + arc unit

Event Timeline

anascime_amd.com retitled this revision from Add support for IRPERF in hwpmc PERF class to hwpmc: Add support for IRPERF in hwpmc PERF class.Thu, Sep 10, 2:42 AM
anascime_amd.com retitled this revision from hwpmc: Add support for IRPERF in hwpmc PERF class to hwpmc: Add support for AMD-only IRPERF in hwpmc PERF class.
anascime_amd.com edited the summary of this revision. (Show Details)
sys/x86/include/specialreg.h
131

Here "HWCR_EFFRFREQCNTMWAIT" should be "HWCR_EFFFREQCNTMWAIT". There's an extra "R". I'll fix this during the reviewing process.

ziaee requested changes to this revision.Thu, Sep 10, 2:33 PM

Hey! Thanks for tagging me. Doc changes looks pretty good, few ideas:

lib/libpmc/pmc.perf.3
11–12

Nd is the "document description". It should be one line. If possible it's better to have it as one rendered line less than 80 columns. It doesn't need to tell them everything, it just needs to tell them if this is the manual they're looking for, and have the correct search keywords. It should definitely not be two lines, I suggest this, which will render like so:

pmc.perf(3) - measurements using the MPERF/APERF/IRPERF frequency counters

105–106

Nit: project uses serial commas per FreeBSD Documentation Primer

187–188

I wouldnt add these lines

This revision now requires changes to proceed.Thu, Sep 10, 2:33 PM

Thanks, @ziaee. I have updated the manual page and fixed the HWCR_EFFRFREQCNTMWAIT typo in specialreg.h.

@ali_mashtizadeh.com @mhorne Could you please take a look at this?

bhyve compatibility observation.

sys/amd64/amd64/initcpu.c
385–391

There is a bhyve compatibility case to account for. On an AMD host that advertises IRPERF and passes this family/model guard, the pinned bhyve/SVM code passes CPUID.IRPERF through, ignores HWCR writes, and does not emulate IRPERF reads. With strict-MSR mode enabled, this read follows a #GP-injection path instead of reaching the non-incrementing-counter fallback.

This could be addressed by correcting bhyve's CPUID exposure until IRPERF is properly virtualized. If changing the probe, rdmsr_safe()/wrmsr_safe() are not direct replacements at this early call site, they require PCPU(CURPCB), which is not ready here during cold initialization. I would preserve the per-CPU boot/resume coverage with any change of the probing phase.

This comes from a guest/host integration case, not a VM boot result or a claim that conforming hardware is broken.

Supporting base sources:
safe helpers,
early BSP ordering,
strict-MSR handling,
CPUID.IRPERF exposure,
HWCR handling and MSR emulation.
https://wiki.freebsd.org/bhyve