riscv: correctly set kernel_pmap hart mask

[mhorne]

Summary

In pmap_bootstrap(), we fill kernel_pmap->pm_active since it is
invariably active on all harts. However, this marks it as active even
for harts that don't exist in the system, which can cause issue when the
mask is passed to the SBI firmware via sbi_remote_sfence_vma().
Specifically, the SBI spec allows SBI_ERR_INVALID_PARAM to be returned
when an invalid hart is set in the mask.

The latest version of OpenSBI does not have this issue, but v0.6 does,
and this is triggering a recently added KASSERT in CI. Change to only setting
bits in pm_active for harts that enter the system.

Test Plan

Boot succeeds with OpenSBI v0.6.

[markj]

Change to only setting bits in pm_active for harts that enter the system.

I don't see where secondary harts set pm_active bits in kernel_pmap. Note that vmspace_pmap(&vmspace0) is not the same thing as kernel_pmap. That is, the pmap_activate_boot() call in init_secondary() doesn't modify kernel_pmap.

[mhorne]

In D27080#604332, @markj wrote:

Change to only setting bits in pm_active for harts that enter the system.

I don't see where secondary harts set pm_active bits in kernel_pmap. Note that vmspace_pmap(&vmspace0) is not the same thing as kernel_pmap. That is, the pmap_activate_boot() call in init_secondary() doesn't modify kernel_pmap.

You're right, I did not read it closely enough. It should be enough to set the active bit directly in init_secondary(), perhaps right before we call pmap_activate_boot() ?

[mhorne]

Activate for secondary cores as well.

[markj]

sys/riscv/riscv/pmap.c

578 ↗

(On Diff #79163)

You might add a sentence or two explaining why we don't use CPU_FILL.