Page MenuHomeFreeBSD

amd64 pmap: block on turnstile for lock-less DI
ClosedPublic

Authored by kib on Jun 14 2019, 12:53 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Nov 19, 12:02 AM
Unknown Object (File)
Wed, Oct 30, 2:15 PM
Unknown Object (File)
Oct 12 2024, 7:01 PM
Unknown Object (File)
Oct 10 2024, 6:39 PM
Unknown Object (File)
Sep 24 2024, 5:28 AM
Unknown Object (File)
Sep 21 2024, 7:52 PM
Unknown Object (File)
Sep 19 2024, 11:59 AM
Unknown Object (File)
Sep 15 2024, 10:00 AM
Subscribers

Details

Summary

Port the code to block on turnstile to lock-less delayed invalidation, instead of yielding. Yield might cause tight loop due to priority inversion.

Since it is impossible to avoid race between block and wake-up, arm 1-tick callout to wakeup when thread blocks itself.

Reported and tested by: mjg

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 24992

Event Timeline

This revision is now accepted and ready to land.Jun 16 2019, 5:17 PM
sys/amd64/amd64/pmap.c
1000

Why do you recheck the loop condition here?

I would suggest that you add a comment to the code explaining why the callout is necessary.

In D20636#446616, @alc wrote:

I would suggest that you add a comment to the code explaining why the callout is necessary.

I will do it when I will be back at my workstation (in about a week).

sys/amd64/amd64/pmap.c
1000

I do it to claim that the only race is due to the lost wakeup. We have in the thread finishing DI:

store generation (atomic, works as seq consistent fence)
load pmap_invl_waiters

Without re-check, the order in the waiting thread is

load generation
increment pmap_invl_waiters (seq cst)

which is reverse order to what is needed to make the increment fence to work against the store generation fence. With the re-check, the order becomes

increment pmap_invl_waiters (seq cst)
load generation

You might argue that the callout would close this race anyway, but callout doing the real work means that the thread lost up to 1 tick of the wall time sitting blocked instead of making progress. So leaving callout use only to wakeup race proper is useful.

sys/amd64/amd64/pmap.c
1000

Thanks. I think this should be explained in a comment.

Add comments explaining the callout use and recheck.

This revision now requires review to proceed.Jun 23 2019, 9:18 AM
sys/amd64/amd64/pmap.c
999

should be "page's"

1000

I would say, "... below the current thread's number. Prepare to block so that we do not waste CPU cycles or worse, suffer livelock."

1004

"... impossible to block without ..."

1007

"... incrementing ..."

1014

"... thread's invalidation ..." (Note the spelling correction to "invlidation".)

kib marked 5 inline comments as done.

Language corrections.

sys/amd64/amd64/pmap.c
1008

Verb tense: "... which will unblock us if we lose ..."

1014

"Re-check the current ..."

Two more language fixes.

This revision is now accepted and ready to land.Jun 23 2019, 8:42 PM
This revision was automatically updated to reflect the committed changes.