Add missing pmap dsb instructions

[andrew]

Summary

We need them in arm64_tlb_flushID when clearing the TLB and after pagetable
stores when moving from an invalid to valid entry. The latter don't call
any of the pmap_invalidate_* functions.

[jchandra]

This fixes the failure on ThunderX2 when INVARIANTS are turned off. The failure was:

panic: vm_fault_hold: fault on nofault entry, addr: 0xffff000040c11000
cpuid = 0
time = 1

Tested with overnight 'make -j 224 buildworld' without failures.
This is more of a 'Tested by' - I don't have enough background here to do a full review. My understanding of ARM spec is you need a dsb after a update of valid pte/pde, before access to the page. ThunderX2 has full out of order execution with weakly ordered loads and stores - so the failure above points to a missing barrier.

[andrew]

In D19097#408644, @jchandra wrote:

Tested with overnight 'make -j 224 buildworld' without failures.

How long does that take?

This is more of a 'Tested by' - I don't have enough background here to do a full review. My understanding of ARM spec is you need a dsb after a update of valid pte/pde, before access to the page. ThunderX2 has full out of order execution with weakly ordered loads and stores - so the failure above points to a missing barrier.

We need one whenever we create a valid page. In many cases there was one because of the call to pmap_invalidate_* functions, however a few were missed.

[jchandra]

In D19097#408663, @andrew wrote:

In D19097#408644, @jchandra wrote:

Tested with overnight 'make -j 224 buildworld' without failures.

How long does that take?

buildworld takes around 30 minutes now, but I don't see the CPUs getting fully loaded. I think this can be improved quite a bit with some profiling.