Allow the workaround_erratum383 to be set from loader

[bz]

Summary

We already turn the workaround on for certain VM_GUEST_VM if specific
CPU features are not present. Some simulation environments, e.g. gem5,
have been found to require more TLB management from the kernel in certain
setups. It is currently unclear why. Turning on the workaround_erratum383
seems to help and make problems (panics) go away.
Given this is a fairly uncommon environment so far, allowing the workaround
to be manually enabled from loader in order to make debugging and comparing
traces easier, but also to allow gem5 run FreeBSD in X86 timing mode, seems
to be the least intrusive option for now until the issue if fully understood.

Sponsored by: DARPA/AFRL

[kib]

x86/x86/mca.c

108 ↗

(On Diff #15897)

I think this is expressed as CTLFLAG_RDTUN in modern days.

[alc]

You should investigate how gem5 handles a situation in which there are two TLB entries (for different page sizes) that map the same virtual address. On real x86 hardware, there are no rules guaranteeing which entry will be chosen, but the translation applied to the virtual address will be based on one of the entries. During promotion (or demotion), we do not normally flush the TLB entries for the affected range of virtual addresses because any duplicate entries should be functionally equivalent. So, we don't care which of the entries is used.

Last year, it was discovered that ARM Cortex-A8 rev 2 processors were generating completely bogus translations in such situations. In other words, the translated physical address was utter garbage.

[bz]

In D6206#132097, @alc wrote:

You should investigate how gem5 handles a situation in which there are two TLB entries (for different page sizes) that map the same virtual address. On real x86 hardware, there are no rules guaranteeing which entry will be chosen, but the translation applied to the virtual address will be based on one of the entries. During promotion (or demotion), we do not normally flush the TLB entries for the affected range of virtual addresses because any duplicate entries should be functionally equivalent. So, we don't care which of the entries is used.

Yeah, I kind-of know what they are doing and had a workaround that made it work, but them talking with kid and reading the SDM I realised what you are saying. http://reviews.gem5.org/r/3401/ I might contact you off-review as I get to it for some more.

[kib]

In D6206#132670, @bz wrote:

http://reviews.gem5.org/r/3401/

We put compatible 4k and 2m entries in the TLB. I.e. for given virtual address, result of the translation to the physical address either with 4k or 2m entry, is the same, and other attributes, like PG_M are also compatible. If this is not true, we have a bug.

[bz]

In D6206#132672, @kib wrote:

In D6206#132670, @bz wrote:

http://reviews.gem5.org/r/3401/

We put compatible 4k and 2m entries in the TLB. I.e. for given virtual address, result of the translation to the physical address either with 4k or 2m entry, is the same, and other attributes, like PG_M are also compatible. If this is not true, we have a bug.

I'll get another full trace and TLB dumps as time permits.

[bz]

Changed it to be RDTUN as @kib suggested.

[kib]

sys/x86/x86/mca.c

107 ↗

(On Diff #16285)

Please put spaces around binary op, i.e.

CTLFLAG_RD | CTLFLAG_RDTUN

[jhb]

sys/x86/x86/mca.c

107 ↗

(On Diff #16285)

Actually, 'CTLFLAG_RDTUN' is 'CTLFLAG_RD | CTLFLAG_TUN', so the CTLFLAG_RD is redundant. This should just be 'CTLFLAG_RDTUN' alone.