I want to keep it independent of the harvestq stuff, as arc4random is now capable of starting quite safely without involving random(9).

See below. look for the ...

chacha_keysetup(&chacha20->ctx, key, CHACHA20_KEYBYTES*8);
chacha_ivsetup(&chacha20->ctx, (u_char *)&tv_now.tv_sec,

... lines.

(My previous reply got lost somehow)

Are you sure about this? Last time I went near critical sections, I got very strong pushback.

Removing the mutexes here also means removing them in other places where there may be data structure conflict. Surely that means putting those in critical sections too? Folks won't like that.

Yeah, that's why I asked for a feedback even if it's a reject :)

But TL;DR really -- I'm fine if we stick using mutex, and we can continue revising the existing status quo at a later time after this change is landed. My main concern is that we could be consuming more bytes than we are supposed to, because there is no restir in the middle of while loop.

My reasoning for critical section:

Basically the idea is to prevent preemption or CPU migrations when we are working on the Chacha20 context (because we are already using a per-CPU context now) and there isn't much point to use a full mutex. The only reason we didn't held a mutex or enter a critical section is because read_random() could potentially block.

Perhaps we should not do a direct restir in line 167 (as it might not belong to us), but set numruns to CHACHA20_RESEED_BYTES + 1 with an atomic store, and all reads to an atomic read [atomic_load_acq_int(&chacha20->numruns)]? This way, the stir part of chacha20_randomstir() would become something like:

getmicrouptime(&tv_now);
atomic_store_rel_int(&chacha20->numruns, CHACHA20_RESEED_BYTES + 1);
critical_enter();
/*
  * We can ONLY do the restir if we are still on the same CPU.
  * Caller is expected to re-check because they have left critical
  * section.
  */
if (chacha20inst[curcpu] == chacha20) {
    chacha_keysetup(&chacha20->ctx, key, CHACHA20_KEYBYTES*8);
    chacha_ivsetup(&chacha20->ctx, (u_char *)&tv_now.tv_sec,
        (u_char *)&tv_now.tv_usec);
    /* Reset for next reseed cycle. */
    chacha20->t_reseed = tv_now.tv_sec + CHACHA20_RESEED_SECONDS;
    chacha20->numruns = 0;
}
critical_exit();