hwpmc: load a started PMC on threads that already run

[afscoelho_gmail.com]

Summary

A process-mode PMC is loaded only at context switch-in. After
pmcstat -t starts a PMC on a process, its running threads keep
running without it. At the end of a time slice the scheduler can pick
the same thread and skip the switch hooks, so a CPU-bound thread alone
on its CPU counts 0 until it blocks.

At PMC_OP_PMCSTART for a process-mode PMC, send the TDA_HWPMC AST to
the running threads of each target. The handler loads the PMCs that
are not loaded on the CPU yet.

Add a test: start a counting PMC on a child that spins and never
blocks, and check that it counts.

Sponsored by: AMD

Test Plan

Start a counting PMC on a CPU-bound process that never blocks:

<cpu-bound loop> & ; pmcstat -t $! -p instructions -w 1 ; kill the loop

Before: DTrace shows no amd_start_pmc_v2 call in the target, and the
final count is 0.
After: the PMC is loaded at start, and the final count is correct.

Note: PMCRW still returns the value saved at the last switch-out, so
the -w interval values stay 0 while such a target does not switch out.
That is a separate issue.

tests/sys/pmc/pmc_attach_running_test fails before and passes after.
Tested with kyua tests/sys/pmc on a kernel with INVARIANTS and WITNESS.

[mhorne]

This one requires a bit more study on my part...

sys/dev/hwpmc/hwpmc_soft.c

398

Note the use of TDA_HWPMC here.

412–422

I have a concern/question about this sharing of the AST handler for these distinct purposes; I am just not sure of the implications.

On the one hand, I believe it is always safe (even useful) to process any pending user callchains. This is why it is registered to run unconditionally; a pending AST from any source will still invoke this handler.

But what about the other case (above), where software PMCs are in use, and ast_sched(curthread, TDA_HWPMC) is called? Is there any downside/danger of invoking pmc_ast_load() on a thread that does not intend it?

[afscoelho_gmail.com]

Rebase on main (D60141 and D60185 are committed). No change.

[afscoelho_gmail.com]

In D60187#1382420, @mhorne wrote:

This one requires a bit more study on my part...

You’re right, soft samples will also call pmc_ast_load(). I think that’s okay because it skips counters already loaded, so it shouldn’t reset or restart them. The callchain capture still runs afterward. The downside I see is the extra check on each soft sample.

[afscoelho_gmail.com]

Rebase only. No code change.