Page MenuHomeFreeBSD

System wide and NUMA domain wide counters support. PMC classes for ARM DMC-620 and CMN-600.
ClosedPublic

Authored by tsoome on May 28 2022, 12:30 PM.

Details

Summary

Add support for system wide and NUMA domain wide counters support.
Add 3 new PMC classes for ARM DMC-620 and CMN-600 controllers PMU.

Sponsored By: ARM
Sponsored By: Ampere Computing

Original review: https://reviews.freebsd.org/D32319

I was asked by Allan to pick up this series of patches while ray is otherwise busy.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

tsoome added a reviewer: pmc.
pauamma added inline comments.
lib/libpmc/pmc.3
203

Unclear to me. Do you mean something like "Counters can be tied/specific to each NUMA domain"?

223

Likewise, do you mean "Counters can have a single value shared among all NUMA domains" here?

Tangentially: since MIPS CPUs are no longer supported per https://www.freebsd.org/platforms/, should they and the associated class still be listed in sys/sys/pmc.h?

In D35342#801259, @pauamma_gundo.com wrote:

Tangentially: since MIPS CPUs are no longer supported per https://www.freebsd.org/platforms/, should they and the associated class still be listed in sys/sys/pmc.h?

I think,it would indeed need cleanup, but I do not want to do it here IMO it should be done in separate commit.

In D35342#801259, @pauamma_gundo.com wrote:

Tangentially: since MIPS CPUs are no longer supported per https://www.freebsd.org/platforms/, should they and the associated class still be listed in sys/sys/pmc.h?

I think,it would indeed need cleanup, but I do not want to do it here IMO it should be done in separate commit.

Fair enough. (I can't do it myself, since I don't have the resources to test kernel or userland changes meaningfully.)

allanjude added inline comments.
lib/libpmc/pmc.3
203

It means that there will be a separate value/counter for each NUMA domain

223

One value for the entire system, rather than one per domain

pauamma requested changes to this revision.Jun 25 2022, 6:41 PM
pauamma added inline comments.
lib/libpmc/pmc.3
203
223
This revision now requires changes to proceed.Jun 25 2022, 6:41 PM
tsoome added inline comments.
lib/libpmc/pmc.3
203

thanks!

223

thanks!

tsoome marked 2 inline comments as done.
tsoome edited the summary of this revision. (Show Details)

Updates to pmc.3 as suggested by pauamma_gundo.com.

Seems to be the same as the original review with my two suggested tweaks? LGTM

Seems to be the same as the original review with my two suggested tweaks? LGTM

Quite so, yes. Since Aleksandr can not complete this work now, we do not want it to get lost.

This revision was not accepted when it landed; it landed in state Needs Review.Jun 26 2022, 9:12 AM
This revision was automatically updated to reflect the committed changes.
gallatin added inline comments.
usr.sbin/pmcstat/pmcstat.c
1190

This utterly breaks system wide profiling for me on all arches that I've tried (amd64, arm64). The cause is that we don't add the pmc for CPU0, as the ID is not invalid. This causes samples from CPU0 to fail to be recognized, and to cause an assertion when analyzing logs:

Core was generated by `pmcstat -R out.pmclog_cpi01 -z 32 -G out.pmcstat_cpi01'.
Program terminated with signal SIGABRT, Aborted.
Sent by thr_kill() from pid 13680 and user 0.
#0 thr_kill () at thr_kill.S:4
4 thr_kill.S: No such file or directory.
(gdb) bt
#0 thr_kill () at thr_kill.S:4
#1 0x00000f8a5e2021a4 in __raise (s=s@entry=0x6) at /data/ocafirmware/FreeBSD/lib/libc/gen/raise.c:52
#2 0x00000f8a5e2b3039 in abort () at /data/ocafirmware/FreeBSD/lib/libc/stdlib/abort.c:67
#3 0x00000f8a5e1e5011 in __assert (func=<optimized out>, file=<optimized out>, line=<optimized out>,
failedexpr=<optimized out>) at /data/ocafirmware/FreeBSD/lib/libc/gen/assert.c:51
#4 0x00000f8236919e09 in pmcstat_analyze_log (args=0xf823691e858 <args>, plugins=0xf823691e120 <plugins>,
pmcstat_stats=0xf823691e980 <pmcstat_stats>, pmcstat_kernproc=0xf8a602a50f0, pmcstat_mergepmc=0x0,
pmcstat_npmcs=0xf823691e9b4 <pmcstat_npmcs>, ps_samples_period=0xf823691e9b8 <ps_samples_period>)
at /data/ocafirmware/FreeBSD/lib/libpmcstat/libpmcstat_logging.c:293
#5 0x00000f823691367a in pmcstat_process_log () at /data/ocafirmware/FreeBSD/usr.sbin/pmcstat/pmcstat_log.c:534
#6 0x00000f8236912fc8 in main (argc=<optimized out>, argv=<optimized out>)
at /data/ocafirmware/FreeBSD/usr.sbin/pmcstat/pmcstat.c:1411
(gdb) frame 4
#4 0x00000f8236919e09 in pmcstat_analyze_log (args=0xf823691e858 <args>, plugins=0xf823691e120 <plugins>,
pmcstat_stats=0xf823691e980 <pmcstat_stats>, pmcstat_kernproc=0xf8a602a50f0, pmcstat_mergepmc=0x0,
pmcstat_npmcs=0xf823691e9b4 <pmcstat_npmcs>, ps_samples_period=0xf823691e9b8 <ps_samples_period>)
at /data/ocafirmware/FreeBSD/lib/libpmcstat/libpmcstat_logging.c:293
293 /data/ocafirmware/FreeBSD/lib/libpmcstat/libpmcstat_logging.c: No such file or directory.