Add support to switch the arm64 EL2 vector after boot

[andrew]

Summary

When loading a hypervisor into EL2 we need to switch the exception
vector base address to one provide by the hypervisor. Allow for this
by providing a way to do this via a hypervisor call instruction.

Obtained from: https://github.com/FreeBSD-UPB/freebsd (earlier version)

[alexandru.elisei_arm.com]

I'm interested in reviewing the patch, I'll try to set aside some time for it.

@andrew, for bhyve on arm64 patches, would you mind adding me as a reviewer with my alexandru.elisei@arm email address from now on?

[andrew]

sys/arm64/arm64/hyp_stub.S

4

@alexandru.elisei_arm.com Are you happy for me to remove All rights reserved? We've been removing them recently as they are no longer needed.

[markj]

sys/arm64/arm64/hyp_stub.S

53

The argument seems to be unused.

60

Typo, haandler

[alexandru.elisei_arm.com]

sys/arm64/arm64/hyp_stub.S

4

Sure, please feel free to remove it, from this file, and from any other files where I'm one of the authors.

[andrew]

• Remove "All rights reserved"
• Fix spelling
• Use the argument to vector_tail

[markj]

sys/arm64/arm64/hyp_stub.S

53

The argument is still there.

[andrew]

sys/arm64/arm64/hyp_stub.S

53

It's used in vector_el1h_sync

[alexandru.elisei_arm.com]

sys/arm64/arm64/locore.S

260

I'm not sure about the purpose of this instruction. I remember copying it from where we disable the MMU from EL1 (after drop_to_el1 returns), but now I can't figure out why this is used. If the MMU is off, it doesn't do anything because data accesses are to Device-nGnRnE memory according to ARM DDI 0487F.b, page D5-2586 (the PE reads and writes directly to main memory).

If the MMU is on, I believe this still has no effect because:

1. Before this point, the kernel hasn't performed any explicit memory accesses, so why do a DSB?
2. I'm speculating, but if this is done as a cheap dcache clean, The PE can speculate data reads and instruction fetches (both are cached in clean cache lines) after the instruction terminates and before we disable the MMU.
3. I'm not 100% sure of this, but the definition of DSB from ARM DDI 0487F.b, page B2-138, only talks about data accesses that come in program order before the barrier, it doesn't say anything about reordering the instruction itself. I think the DSB can be reordered at any point in the program as long as it allowed by the barrier definition.

263

I haven't been reading the code, but I haven't found where the dcache is invalidated before the MMU is turned on at EL1. Where is that done?

325

According to the definition of a context synchronization event from ARM DDI 0487F.b, page Glossary-8112, the isb and eret instructions are equivalent. I believe this isb is redundant.

[andrew]

sys/arm64/arm64/locore.S

263

In the normal boot process we expect the bootloader to have invalidated the dcache

[andrew]

Remove an unneeded dsb

[alexandru.elisei_arm.com]

sys/arm64/arm64/locore.S

263

The patch itself looks alright to me, but I haven't tested it.

As for why I think FreeBSD should do dcache invalidation before turning the MMU back on, wall of text below.

I've been reading the Arm ARM ARM DDI 0487F.b trying to find out exactly what a CPU is allowed to speculate, and I've only found what is *not* able to speculate (section B2.3.6 "Restrictions on the effects of speculation"). From that I assume the CPU is able to speculate *any* loads regardless of the program order, and modify the dcache as a result. This gets somewhat confirmed by the definition of the SB (Speculation Barrier) instruction (page C6-1188):

"In particular, any instruction that appears later in the program order than the barrier cannot cause a speculative allocation into any caching structure where the allocation of that entry could be
indicative of any data value present in memory or in the registers."

My interpretation of that statement is that without the barrier, the PE is allowed to modify the dcache as a result of speculation.

This is the definition of Device memory from page B2-117:

"The Arm architecture forbids Speculative reads of any type of Device memory. This means Device memory types are suitable attributes for read-sensitive Locations."

When address translation is disabled, data accesses are to Device-nGnRnE memory type (page D5-2586).

The point I am trying to make is that as long as the MMU is enabled (SCTLR_ELx.M == 1), the PE is allowed to speculate any loads and modify the dcache. This is something that FreeBSD allows the bootloader to do.

The other side of the story is that even if FreeBSD starts execution with the MMU disabled, if it's running under a hypervisor, the dcaches might have been populated while the host was running on the same PE (either intentionally or as a result of speculation). As a result, dcache invalidation must be performed right before the MMU is turned on (after the last load in program order, to be more precise), so any speculative reads on the host side will populate the dcache with the latest values that the guest wrote to memory.

I hope what I wrote above makes sense.

[andrew]

Rebase after locore.S changes

[andrew]

sys/arm64/arm64/locore.S

263

Which memory do you think needs d-cache management? As far as I can tell we should invalidate the d-cache for the page tables as there may be data for them in the cache, however we create them with the cache disabled.

[andrew]

Remove an unneeded instruction