Details

Reviewers

mhorne
ali_mashtizadeh.com
netchild
afscoelho_gmail.com
anascime_amd.com
davichazbh_gmail.com

Summary

Add ATF tests for hwpmc(4). Vendor-neutral tests go in tests/sys/pmc;
tests that depend on AMD hardware interfaces go in tests/sys/pmc/amd,
which is only built on amd64.

tests/sys/pmc:

pmc_pidtrack_test      process-mode PMCs follow their PID: a live
                       self-read counts the process' own work, an idle
                       process sharing the CPU with a busy one does not
                       pick up its work, and the count survives a
                       context switch
pmc_callchain_ur_test  pmcstat -U collects user callchains through the
                       PMC_UR ring; regression test for the KASSERTs
                       fixed in rGaf3929c5152b and rG66118c3f1011
pmc_cli_test           pmc(8): missing log, list-events, record/info
                       round trip with the "instructions" alias
pmcstat_tsc_test       (amd64) tsc_freq in the initlog record, TSC
                       column of pmcstat -R, tsc in pmc filter -j,
                       pmcstat -G callgraph
pmc_rapl_test          (amd64) RAPL class: 64-bit counters, system
                       mode only, PKG and CORES energy increase under
                       load

tests/sys/pmc/amd:

pmu_detect_test        K8 class with 48-bit counters, at least six core
                       counters, TSC class alongside it
pmu_event_test         pmu-events lookup and EventSelect/UnitMask/OS/USR
                       encoding checked against the amdzen1..amdzen6
                       tables; ex_ret_instr counts a known loop
pmu_event_neg_test     raw PMCALLOCATE rejects an otherwise valid config
                       word with HOST, GUEST, HOST|GUEST, reserved bit
                       44 or PRECISERETIRE without PMC_CAP_PRECISE
                       (pmc_exterr_test covers an all-ones config word)
pmu_mux_test           Zen 4: six core counters can be held at once, a
                       seventh is refused, a released one is reusable
rapl_domains_test      AMD RAPL has at most PKG and CORES rows and no
                       DRAM domain
pmc_frontend_test      pmc(8) frontend study on IBS fetch samples
amd_pmc_test_common.h  CPUID-based Zen generation detection and helpers

Tests for PMC grouping and for IBS decoding depend on changes that are
not in main and will be posted together with those changes.

Sponsored by: Advanced Micro Devices, Inc.
Signed-off-by: Osvaldo Janeri Filho <ojanerif@amd.com>

Test Plan

Applied to main 3171a60de403 (2026-09-30), buildworld and buildkernel
GENERIC (INVARIANTS, WITNESS), installworld, booted on an AMD EPYC 9654
(Zen 4, 192 CPUs), kldload hwpmc, kyua test in /usr/tests/sys/pmc:
86/86 passed in three consecutive runs (46 existing and 40 new test
cases), no panic and no WITNESS warning.

Diff Detail

Lint

Lint Skipped

Unit

Tests Skipped

Event Timeline

janeri_gmail.com requested review of this revision.Wed, Sep 9, 9:06 PM

janeri_gmail.com created this revision.

janeri_gmail.com updated this revision to Diff 186763.Tue, Sep 15, 10:40 AM

End of last month I added some tests/sys/pmc tests. While the items in this review seem to be AMD specific, they are testing parts of the PMC subsystem. I would expect the tests in the pmc directory (if you want in an amd subdirectory, but I have already some amd specific tests in the main OMC directory), than as it is vice versa.
Is it feasible to use speaking names for model in amd_umcdf_map_zen()?
hwpmc_exterr_test.c is a near-copy of the in-tree tests/sys/pmc/pmc_exterr_test.c. 13 of 14 case names identical; adds amd_missing_pmu_flag, drops amd_invalid_df_config_bits. Copyright/author line changed from the original.
The directory never builds. tests/sys/Makefile gains TESTS_SUBDIRS+= ${_amd} but _amd is defined nowhere in the diff (the _pmc/_audit pattern needs the .if … _amd= amd .endif block, and _pmc is also gated on MK_PMC, which this isn't).
tests/sys/amd/pmc/Makefile starts with a UTF-8 BOM and has no trailing newline. make may choke on the BOM.
Content for other diffs shipped here. msr_snapshot.[ch] is not compiled ("see follow-on diff").
The two umcdf/ headers seem to belong to to something that isn't in this diff, yet five pmc tests #include "../umcdf/amd_umcdf_common.h" for Zen-generation decoding. Not sure if the decde should belong together with amd_pmc_test_common.h or with this something else not included here.
hwpmc_rdpmc_test is in the Makefile and the diff but not in the summary.
Roughly half the suite is not AMD-specific. The callchain-UR regression (af3929c5152b, 66118c3f1011 — both vendor-neutral hwpmc_mod.c fixes), pmcstat TSC column, group-alloc atomicity, pidtrack, pmc(8) CLI smoke: they only use AMD event names. Under tests/sys/pmc/ with a per-vendor event table they'd cover Intel too. Having this in a vendor-specific subdirectory would need a reorg to add intel support. Better to not use a vendor specific directory (I do not ask someone from AMD to provide the Intel parts, but not making it harder to add them would be polite).
Vendor detection via /dev/cpuctl0 (needs cpuctl.ko + root) in the C tests vs. kern.hwpmc.cpuid in the sh tests. <machine/cpufunc.h> do_cpuid() needs no device.
AMDID2_* redefined instead of <machine/specialreg.h>.
"../amd_pmc_test_common.h" relative includes and -I${SRCTOP}/tests/sys/amd — one is redundant.

I also suggest per-file arch gating and runtime vendor skips.

Thank you for the thorough review.

Addressed in the updated diff:

Moved vendor-neutral tests to tests/sys/pmc/: pmc_pidtrack_test.c, pmcstat_tsc_test.sh, pmc_callchain_ur_test.sh. The pidtrack test uses find_counting_event() for portability; the callchain-UR test retains its amd64 uname guard internally.

Removed hwpmc_exterr_test.c -- rGe555692d1bbb already covers those cases. If any AMD/IBS cases from our version are not in the landed file we will submit them as an incremental patch to pmc_exterr_test.c separately.

Added hwpmc_rdpmc_test.c to tests/sys/amd/pmc/ (RDPMC userspace validation: pmc_get_msr() ECX encoding, CR4.PCE grant, rdpmc vs pmc_read() agreement).

amd_test_do_cpuid() rewritten to use do_cpuid() from <machine/cpufunc.h>; no /dev/cpuctl0, no cpuctl.ko dependency. Removed <sys/cpuctl.h>, <sys/ioctl.h>, <fcntl.h> from the header.

Removed AMDID2_PNXC and AMDID2_PTSCEL2I redefinitions from amd_pmc_test_common.h -- both are in <machine/specialreg.h>.

Replaced relative #include "../amd_pmc_test_common.h" with #include "amd_pmc_test_common.h" throughout; -I${SRCTOP}/tests/sys/amd in the Makefile resolves it. Same for the umcdf header.

Removed msr_snapshot.[ch] from this diff. Those files are used by hwpmc_grouping_test, which depends on the PMC grouping kernel API currently pending upstream. We will address msr_snapshot together with the grouping tests once that API merges.

Added product names as inline comments to amd_umcdf_map_zen() (Pinnacle Ridge, Rome, Matisse, Genoa, Turin, etc.).

Not yet addressed (pending decisions or separate diffs):

Regarding the suggestion to colocate under tests/sys/pmc/amd/ instead of tests/sys/amd/pmc/: the AMD-specific tests (RAPL, pmu-events, IBS errata, RDPMC, grouping) are tightly coupled to AMD hardware interfaces and would be difficult to generalize. We kept the vendor-specific directory for those and moved the portable tests to tests/sys/pmc/ as suggested.

The two umcdf/ headers are shared with the follow-on umcdf test suite. We can extract the Zen-generation decode helpers into a smaller header under amd/ if that is preferred over shipping the full umcdf header here.

Please update to a recent version of the main branch. You created a new pmc/Makefile, but the tip of main has one already. As-is the patch does not apply, and is not self-sufficient. It also contains a build-host hack (include dir and lib dir pointing hardcoded to /usr/local, whereas ATF should be in the base and doesn't need that. I have seen several internal Jira IDs. Not sure if you want to have that in public. And tests are skipped unless amd.pmc.grouping.runtime is set (which nothing does in this patch). umcdf/amd_umcdf_common.h still uses /dev/cpuctl0 (do_cpuid() went into amd_pmc_test_common.h only). The family/model messages and the AuthenticAMD-25-00-1 fixture names still print hex where the map now has the names.

I can understand from a vendor point of view, to have all related code together. I'm grateful that AMD is sponsoring this work. Please keep in mind that we do not have a vendor directory in sys/ and that our organisation in sys/ is functional. Having amd-only stuff in subdirectories or in the same directory as the subsystem is in my opinion a personal-taste question (at least here). So I don't care if it is pmc/amd/test1.x or pmc/amd_test1.x, but sys/amd/ looks wrong.

Reviews target a commit, and the patch here in this review provides parts which don't get used, references parts which are not included, and does not seem to build/test everything you want it to build/test. I suggest to update to a recent main branch, apply the subset of the bigger picture you want to have reviewed there, cut the diff, apply to to a fresh copy of main, and check if it does all what you want it to do (= check if it is self-contained, and complete).

Updated diff against current FreeBSD main (43347e8).

Structural change: moved everything from tests/sys/amd/ to
tests/sys/pmc/amd/ per your feedback. Patch applies cleanly
against main and is self-contained.

Also fixed in this revision:

do_cpuid() in amd_umcdf_common.h (missed in v2)
removed remaining internal ticket references
tests/sys/Makefile adds _amd= amd gated on amd64, following the _pmc/_audit pattern
tests/sys/pmc/Makefile adds SUBDIR+= amd

Makefiles:

tests/sys/Makefile descends into tests/sys/amd, which this diff no longer creates.
tests/sys/pmc/Makefile says SUBDIR+= amd. bsd.test.mk reserves SUBDIR for helper/data dirs; only TESTS_SUBDIRS gets include("amd/Kyuafile") in the parent Kyuafile, so as posted kyua test from /usr/tests are not run. And it's ungated: MK_PMC is default-on everywhere, the C tests need do_cpuid() from <machine/cpufunc.h> (amd64/i386 only) → aarch64/riscv/powerpc break. The architecture guard needs to be here.
pmc_pidtrack_test is a .c file listed under ATF_TESTS_SH. atf.test.mk makes it depend on pmc_pidtrack_test.sh, which doesn't exist. ATF_TESTS_C is the correct one.

The three vendor-neutral tests (pidtrack, tsc, callchain-UR) that the previous patch had rightly placed in tests/sys/pmc/ came back into the vendor directory.

You have two "QA branch not installed?" texts in pmcstat_tsc_test.sh. This does not make sense here in FreeBSD.

v9: address all reviewer feedback

pmc_pidtrack_test.c: move uint64_t out[3] declaration before any executable statement in pidtrack_ctxswitch body (KNF C89, mhorne)
pmc_pidtrack_test.c: fork(idle)<0 error path now closes all six descriptors including busyfd[1] and startfd[0] (mhorne)
pmc_callchain_ur_test.sh: replace private commit hashes with descriptive text (netchild)
pmcstat_tsc_test.sh: remove empty () placeholders from comments and trim duplicate phrase (netchild)
amd/Makefile: strip CI-only block with hardcoded developer path; this block must not appear in the upstream tree (netchild)
pmu_mux_test.c: fix brace alignment in cleanup for-loop, tighten Zen4-only guard to zen != ZEN_4 (ali)
rapl_test.c: reorder ATF_CHECK_MSG guard before fprintf; guard unsigned subtraction (mhorne)
Author lines in shell scripts: full name + email format
amd_pmc_test_common.h: add Copyright line

Tested on AMD EPYC 9654 (Zen 4, 192 logical CPUs),
FreeBSD 16.0-CURRENT HWPMC_DBG kernel:

44/51 passed, 0 failed, 0 broken, 7 skipped (all expected)

please ignore older comments.

addressed all items raised:

tests/sys/Makefile: removed from this diff; wiring goes through tests/sys/pmc/Makefile only
tests/sys/pmc/Makefile: SUBDIR changed to TESTS_SUBDIRS; added .if ${MACHINE_ARCH} == "amd64" guard so aarch64/riscv/powerpc are not affected
pmc_pidtrack_test: moved from ATF_TESTS_SH to ATF_TESTS_C
Vendor-neutral tests (pmc_pidtrack_test, pmcstat_tsc_test, pmc_callchain_ur_test) are in tests/sys/pmc/ directly
"QA branch not installed?" text removed
amd/Makefile: CI-only block with hardcoded developer path removed
pmc_pidtrack_test.c: uint64_t out[3] declaration moved before any executable statement (KNF C89)
pmc_pidtrack_test.c: fork(idle)<0 error path closes all six fds
pmu_mux_test.c: brace alignment fixed; Zen4-only guard tightened
rapl_test.c: ATF_CHECK_MSG guard reordered before fprintf
Author lines in shell scripts: full name + email format

Tested on AMD EPYC 9654 (Zen 4, 192 logical CPUs),
FreeBSD 16.0-CURRENT with hwpmc(4):

44/51 passed, 0 failed, 0 broken, 7 skipped (all expected).

Tested Diff 187278 on an AMD Ryzen 5 5600X (Zen 3, Family 19h,
Model 21h, Stepping 0, Id 0xa20f10, 12 threads), FreeBSD
16.0-CURRENT at 10a3562a019b, kernel and world in sync.

Applying: "arc patch D59549" fails (the new files come out as
modifications of nonexistent files), but the raw diff applies
cleanly with "git apply".

Build/install issues:

tests/sys/pmc/pmc_pidtrack_test.c:282: kill() is undeclared; <signal.h> is not included. Fatal with -Werror.

tests/sys/pmc/pmc_pidtrack_test.c:386: "uint64_t out[3]" in the child branch shadows the declaration at line 357 (-Wshadow, fatal with -Werror). Since the child works on its own copy after fork(), dropping the inner declaration should be enough.

etc/mtree/BSD.tests.dist is not updated, so /usr/tests/sys/pmc/amd is not created and "make install" fails. installworld would fail the same way.

pmcinfo_thread_count is installed to /pmcinfo_thread_count (debug file to /.debug/), apparently because it has no install directory in tests/sys/pmc/amd/Makefile.

Because of 1 and 2, the vendor-neutral tests in tests/sys/pmc
were not run. For the AMD suite, after creating the directory by
hand: 51 total, 27 passed, 24 skipped, 0 failed, 0 broken.

Skips:

6 grouping tests: opt-in, as documented.
4 pmu_mux_test: Zen 4 only; expected, no CPUID Fn8000_0022 here.
7 pmu_event_test and 6 pmu_event_neg_test skip with "CPU is Zen 3; requires Zen4+". Zen 3 has pmu-events tables (pmccontrol -L lists ex_ret_instr, l3_lookup_state.*, etc. on this machine), so lookup by name, os/user and umask encoding, and the e2e count look applicable to Zen 3 as well. Would it be possible to gate only the Zen 4 specific cases (EventSelect[11:8])?
pmcstat_ibs_errata_test skips with "erratum #1238 suppression not present". It uses synthetic pmclog fixtures, so the result does not depend on the host CPU: it skips because pmcstat in main lacks the suppression. The summary describes it as skipping on non-affected CPUIDs. Maybe it belongs with the tests that depend on pending changes?

Happy to rerun on Zen 3 after an update.

ojanerif_amd.com updated this revision to Diff 187767.Sat, Sep 26, 6:52 PM

Herald added a subscriber: ivy. · View Herald TranscriptSat, Sep 26, 6:52 PM

Thank you for the report @paulo_nlink.com.br

Fixed:

<signal.h> missing — adding the include
out[3] shadow — removing the inner child-block declaration
pmcinfo_thread_count install path — PROGS_TESTSDIR+= instead of PROGS+=
BSD.tests.dist — adding /usr/tests/sys/pmc/amd entry
LIBADD.pmcinfo_thread_count+= pmc — explicit per-target link flag

For the Zen 3 skips: pmu_event and pmu_event_neg SMOKE and E2E tests
require the amdzen4 pmu-events JSON tables shipped with hwpmc (there
are no Zen 3 equivalents in-tree yet).
The UNIT tests (pure bit-manipulation, no hardware) already
run on any CPU including Zen 3.

tests/sys/pmc/Makefile drops existing tests, looks like an updated tree got only your makefile based on an older version
similar for etc/mtree/BSD.tests.dist (loses changes from Sept 1-5)
the Kyuafiles are not supposed to be committed (generated), so they should not be in the diff. I suspect an in-src-tree build instead of "make objdir" before the build.
The helper pmcinfo_thread_count is not built. It moved from PROGS+= to PROGS_TESTSDIR+=, and nothing in share/mk reads that variable. The in-tree way is BINDIR= ${TESTSDIR} plus PROGS+=, as in tests/sys/cam/ctl/Makefile. That missing BINDIR is probably why Paulo's build installed it to /.

There is lib/libpmc/pmu-events/arch/x86/amdzen3/, which looks like it should match Paulo's CPU. Is this something else?

Thanks for the reviews. Regenerated against current main and uploaded
with full context, so "arc patch D59549" should work now:

Makefile / BSD.tests.dist: the previous diff was generated against a stale tree. Only new lines are added now.
Kyuafiles removed.
pmcinfo_thread_count: my mistake, PROGS_TESTSDIR does not exist. The helper was only used by the grouping atomicity test, which is out of this diff now.
amdzen3: you are right, my earlier reply was wrong. pmu_event now has expectations per pmu-events table (Zen 1 to Zen 6), and pmu_event_neg does not depend on the generation.
Vendor-neutral tests are in tests/sys/pmc: pidtrack, the callchain-UR regression, pmcstat TSC, the pmc(8) CLI cases and the RAPL class tests. Only the AMD-specific parts stay in pmc/amd.
Out of this diff: the grouping and IBS decoding tests (they depend on changes not in main), the umcdf headers, hwpmc_rdpmc_test and pmu_cross_class_alloc_test.

Tested as described in the test plan: 86/86 on main 3171a60de403 with
INVARIANTS and WITNESS.

@paulo_nlink.com.br, would you mind re-running on your Zen 3?

tests/sys/pmc: add pidtrack, callchain, pmc(8), RAPL and AMD tests
Needs ReviewPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 186968

b/tests/sys/Makefile

b/tests/sys/pmc/Makefile

b/tests/sys/pmc/amd/Makefile

b/tests/sys/pmc/amd/amd_pmc_test_common.h

b/tests/sys/pmc/amd/pmc_callchain_ur_test.sh

b/tests/sys/pmc/amd/pmc_cli_smoke_test.sh

b/tests/sys/pmc/amd/pmc_pidtrack_test.c

b/tests/sys/pmc/amd/pmcinfo_snapshot.h

b/tests/sys/pmc/amd/pmcinfo_snapshot.c

b/tests/sys/pmc/amd/pmcinfo_thread_count.c

b/tests/sys/pmc/amd/pmcstat_group_alloc_atomicity_test.sh

b/tests/sys/pmc/amd/pmcstat_grouping_test.sh

b/tests/sys/pmc/amd/pmcstat_ibs_errata_test.sh

b/tests/sys/pmc/amd/pmcstat_tsc_test.sh

b/tests/sys/pmc/amd/pmu_detect_test.c

b/tests/sys/pmc/amd/pmu_event_neg_test.c

b/tests/sys/pmc/amd/pmu_event_test.c

b/tests/sys/pmc/amd/pmu_mux_test.c

b/tests/sys/pmc/amd/rapl_test.c

b/tests/sys/pmc/amd/umcdf/amd_umcdf_common.h

b/tests/sys/pmc/amd/umcdf/amd_umcdf_decode.h

tests/sys/pmc: add pidtrack, callchain, pmc(8), RAPL and AMD testsNeeds ReviewPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 186968

b/tests/sys/Makefile

b/tests/sys/pmc/Makefile

b/tests/sys/pmc/amd/Makefile

b/tests/sys/pmc/amd/amd_pmc_test_common.h

b/tests/sys/pmc/amd/pmc_callchain_ur_test.sh

b/tests/sys/pmc/amd/pmc_cli_smoke_test.sh

b/tests/sys/pmc/amd/pmc_pidtrack_test.c

b/tests/sys/pmc/amd/pmcinfo_snapshot.h

b/tests/sys/pmc/amd/pmcinfo_snapshot.c

b/tests/sys/pmc/amd/pmcinfo_thread_count.c

b/tests/sys/pmc/amd/pmcstat_group_alloc_atomicity_test.sh

b/tests/sys/pmc/amd/pmcstat_grouping_test.sh

b/tests/sys/pmc/amd/pmcstat_ibs_errata_test.sh

b/tests/sys/pmc/amd/pmcstat_tsc_test.sh

b/tests/sys/pmc/amd/pmu_detect_test.c

b/tests/sys/pmc/amd/pmu_event_neg_test.c

b/tests/sys/pmc/amd/pmu_event_test.c

b/tests/sys/pmc/amd/pmu_mux_test.c

b/tests/sys/pmc/amd/rapl_test.c

b/tests/sys/pmc/amd/umcdf/amd_umcdf_common.h

b/tests/sys/pmc/amd/umcdf/amd_umcdf_decode.h

tests/sys/pmc: add pidtrack, callchain, pmc(8), RAPL and AMD tests
Needs ReviewPublic
Actions

Revision Contents
Changeset List