hwpmc: stop process PMCs on exit with PerfMonV2

[afscoelho_gmail.com]

Summary

Since 9d3064010590, pmc_process_exit() calls pmc_process_csw_stop_all()
first. That clears pps_cpustate for every virtual PMC on the CPU, so the
exit loop then skips the stop and the final read.

On AMD PerfMonV2 this has two effects:

• The count from the last time slice is lost. A process that never switches out before exit reports 0 with pmcstat -p.
• The EVSEL enable bit stays set. After kldunload, the next kldload sees it and says "PMCs maybe in use by firmware!".

When the class has pcd_stop_all, always stop and read the PMC, as
pmc_process_csw_out() already does.

Fixes: 9d3064010590 ("hwpmc_amd: add PerfMonV2 global-control path")
Sponsored by: AMD

Test Plan

On a Zen3 system (kern.hwpmc.amd_perfmon_v2=1):

pmcstat -p instructions sh -c 'i=0; while [ $i -lt 300000 ]; do i=$((i+1)); done'

Before: prints 0. After 25 runs, 25 PERF_CTL MSRs still have EN (bit 22)
set, and reloading hwpmc loads only the SOFT class.

After: prints about 4.65e9, the same as a system-mode PMC for the same
work. No PERF_CTL has EN set, and hwpmc reloads with all classes.

[mhorne]

My question is: why does pmc_process_csw_stop_all() mess with the pps_cpustate fields in the first place? Why not leave it to this code, whose purpose is to handle this?

[afscoelho_gmail.com]

Thanks, good point. It shouldn't. I added it in the PerfMonV2 change (D58256) to keep the "clear, then stop" order while the global gate closes. You're right, that isn't needed. I updated the diff to remove pmc_process_csw_out_prepare(), so the exit loop is back to how it was. I re-tested on Zen3 and got the same good results as the first fix. Thanks for catching this!

[mhorne]

LGTM! Please update the change description and I can merge later today.