amd64: Initialize IPI scoreboard earlier

[kib]

Summary

Scoreboard is needed a moment when smp_started == true.  If some kernel
daemon thread is started before scoreboard is inited, and does some pmap
operation that requires TLB maintanence, which races with SMP startup,
we might dereference NULL invl_scoreboard.  This is particularly easy
to trigger when EARLY_AP_STARTUP is not defined.  I do not think we start
any kernel thread before SMP startup when EARLY_AP_STARTUP is defined.

Reported by:    glebius

[glebius]

You gave me the patch in December 2021 and I was running with it for a while. I think all the time until SMR inpcb branch has been merged to main. After that I lost the patch in my rebases, so has been running without it last half a year. Didn't see the panic, though. But as you remember the panic is quite rare. I think the patch should go in as long as logically it is correct. I'm no expert in the area at all.

[markj]

I do not think we start any kernel thread before SMP startup when EARLY_AP_STARTUP is defined.

FWIW, the audit subsystem starts one at SI_SUB_AUDIT. There could be others.

sys/amd64/amd64/mp_machdep.c

582

start_aps() also uses SI_SUB_SMP/SI_ORDER_FIRST. Can't we just initialize the scoreboard at SI_SUB_SMP - 1 to solve the problem?

[kib]

Use SI_SUB_SMP - 1.

[markj]

sys/amd64/amd64/mp_machdep.c

582

Does not have to be _FIRST.

[kib]

sys/amd64/amd64/mp_machdep.c

582

I considered changing it to _ANY, but really do not see a difference.
Do you insist? Or do you mean something else?

[markj]

sys/amd64/amd64/mp_machdep.c

582

There's no difference, but whenever I see _FIRST is used, I assume there is some specific reason for it. _ANY is effectively a default. So, it is just a bit confusing IMO. I don't insist though.