vmm/svm: iopm_bitmap and msr_bitmap must be contiguous in physical memory

[avg]

Summary

To achieve that the whole svm_softc is allocated with contigmalloc now.
It would be more effient to de-embed those arrays and allocate only them
with contigmalloc.

Previously, if malloc(9) used non-contiguous pages for the arrays, then
random bits in physical pages next to the first page would be used to
determine permissions for I/O port and MSR accesses. That could result
in a guest dangerously modifying the host hardware configuration.

One example is that sometimes NMI watchdog driver in a Linux guest
would be able to configure a performance counter on a host system.
The counter would generate an interrupt and if hwpmc(4) driver is loaded
on the host, then the interrupt would be delivered as an NMI.

Test Plan

Tested locally for a couple of days with multiple boots of CentOS and Gentoo
guests. No regressions so far.

[avg]

Just a little bit of additional information.

The start of the story is here: https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-June/003644.html

This is what made me look at MSR permissions:

Mar 16 10:14:42 gentoo kernel: [    0.198926] Failed to access perfctr msr (MSR c0010007 is 0)
Mar 16 11:23:24 gentoo kernel: [    0.206671] Failed to access perfctr msr (MSR c0010007 is ffff)
Mar 16 13:25:32 gentoo kernel: [    0.199258] Failed to access perfctr msr (MSR c0010007 is 5a980a0f3654)
Mar 17 15:28:37 gentoo kernel: [    0.302609] Failed to access perfctr msr (MSR c0010007 is 0)
Mar 17 17:44:13 gentoo kernel: [    0.198270] Failed to access perfctr msr (MSR c0010007 is 0)
Mar 21 09:52:21 gentoo kernel: [    0.243897] Failed to access perfctr msr (MSR c0010007 is 0)
Mar 21 23:09:32 gentoo kernel: [    0.210777] Failed to access perfctr msr (MSR c0010007 is 0)
Mar 25 20:16:02 gentoo kernel: [    0.248861] Failed to access perfctr msr (MSR c0010007 is 0)
Sep 15 10:50:03 gentoo kernel: [    0.248273] Failed to access perfctr msr (MSR c0010007 is 0)
Sep 20 19:03:51 gentoo kernel: [    0.245672] Failed to access perfctr msr (MSR c0010007 is dfff3fb7b17e)
Oct 21 12:07:15 gentoo kernel: [    0.241297] Failed to access perfctr msr (MSRc0010007 is 0)

As you can see, most of the time c0010007 is correctly read as zero, but not always.
This is what happens when the protection works:

Performance Events: Broken PMU hardware detected, using software events only.
NMI watchdog: disabled (cpu0): hardware events not enabled

And this is what happened the last time I got the NMI on the host:

Performance Events: AMD PMU driver.
... version:                0
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             00007fffffffffff
... fixed-purpose events:   0
... event mask:             000000000000000f
MCE: In-kernel MCE decoding enabled.
x86: Booted up 1 node, 1 CPUs
smpboot: Total of 1 processors activated (6420.42 BogoMIPS)
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.

And I was able to confirm that one of the performance counters was active on the host:

$ cpucontrol -m 0xc0010000 /dev/cpuctl1
MSR 0xc0010000: 0x00000000 0x00530076

I interpret this as event 0x76 'CPU clocks not halted'. Enabled for both userland and kernel modes. Level detect. Counter enabled, APIC interrupt enabled.

[grehan]

Great catch.