amd64: Implement per-thread kernel stack canaries

[bnovkov]

Summary

This change introduces per-thread kernel stack canaries using
clang's -mstack-protector-guard=tls flag. Under this scheme, we
instruct the compiler to fetch the canary value through the gs
register, allowing us to store the canary value in struct pcpu.

N.B.: This revision depends on LLVM PR 213107 since clang will not generate the proper gs-aware instrumentation on FreeBSD.

Test Plan

The patch survives a full run of the regression test suite.

[markj]

Nice! Mostly looks good, the only substantive comment is about the build integration.

sys/amd64/amd64/cpu_switch.S

153

I guess this is just defensive zeroing? The register will be clobbered below anyway. I'm not sure it's worth bothering with this.

sys/amd64/amd64/vm_machdep.c

204

You're using a mixture of styles, #ifdef PERTHREAD_SSP and #if defined(PERTHREAD_SSP), better to be consistent.

sys/conf/Makefile.amd64

43

and below

45

Probably this all needs to be conditional on MK_SSP != "no"?

Is there a good way to automatically disable PERTHREAD_SSP at compile time if MK_SSP != "no", for users that want to opt out of SSP entirely?

sys/conf/options.amd64

13

This sorts before PV_STATS.

[kib]

sys/amd64/amd64/locore.S

88

Clearing %rbx does not make sense, it is the same value as in boot_canary.

sys/amd64/include/proc.h

80

Why the type is uintptr_t? Is it supposed to be the machine word size? If yes, then either register_t or an explicit u_long is better IMO.

[bnovkov]

Address @markj 's and @kib 's comments

[bnovkov]

sys/conf/Makefile.amd64

45

Is there a good way to automatically disable PERTHREAD_SSP at compile time if MK_SSP != "no", for users that want to opt out of SSP entirely?

I am not sure honestly, and it looks like this currently isn't possible on arm64 as well.
I'll look into kern.opts.mk and see if something can be done there.

[markj]

sys/conf/Makefile.amd64

45

I think the main source of overhead in that configuration would be the arc4random() call that happens during thread creation.

I'm curious about how expensive that is. A while ago I added a pthread_create() benchmark to will-it-scale: https://github.com/markjdb/will-it-scale/blob/master/tests/pthread_create1.c and I wonder how much the PERTHREAD_SSP option affects it.

[kib]

sys/amd64/amd64/cpu_switch.S

34

Is this really needed? I suspect that the right change would be to force inclusion of opt_global.h using the compiler switch for .S, if not already done.