Page MenuHomeFreeBSD

Enabling FreeBSD on ARM64 Hyper-V
Needs ReviewPublic

Authored by schakrabarti_microsoft.com on Fri, Aug 5, 10:46 PM.

Details

Reviewers
whu
manu
Summary

This patch enables FreeBSD to get installed on ARM64 Hyper-V.
Files in aarch64 directory are specific for arm64, and new x86 directory contains files which are for both amd64 and i386.
vmbus.c and hyperv.c has been refactored and parts of the code has been moved to vmbus_aarch64.c and hyperv_aarch64.c for arm64.
Same way vmbus_x86.c and hyperv_x86.c contains same functions for x86.
vmbus.c and hyperv.c contains common code, and to keep similarity, in x86 or in aarch64 some stub functions have been placed.
hyperv_reg.h contains MSR details, in aarch64 we have replaced some of the MSR values with ARM64 Hyper-V specific MSR values.
hyperv_machdep.c contains new hypercall implementations for HvCallGetVpRegisters and HvCallSetVpRegisters using arm smccc hvc abi.

I have introduced new arm smccc hvc 1.2 abi for HvCallGetVpRegisters, as it needs to access beyond x0-x3 registers. The respective assembly code is in smccc_1_2_arm64.S.

Test Plan

The basic functionality of FreeBSD is working on ARM64 Hyper-V.
The change has not caused any regression for X86 Hyper-V.

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

emaste added inline comments.
share/mk/src.opts.mk
334

We'll want to update the comment also, or more likely just delete it as it's clearly limiting HyperV to some subset of architectures.

stand/efi/loader/conf.c
90

This seems odd, paging @imp

sys/Makefile
22 ↗(On Diff #108915)

this is an extraneous diff I think

sys/arm/conf/GENERIC
285 ↗(On Diff #108915)

Is 32-bit ARM supported?

sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c
2

May want to add 2022 here?

This change is kinda large to review... Please see if you can break it up into smaller pieces with better explanations for each of the steps.
I've only just started reviewing things... I'm not super familiar with arm64 hypervisor interfaces...

This is super cool...

stand/efi/loader/conf.c
90

This seems wrong, and the #define is weird too...
So I'd like more details...
Because we don't build stand with that define, as far as I know, normally.

sys/conf/files.x86
135

This split out suggests a good way to make this smaller...

sys/dev/hyperv/storvsc/hv_storvsc_drv_freebsd.c
1842 ↗(On Diff #108915)

why's that?

1861 ↗(On Diff #108915)

I don't think this #ifdef is needed. This will be a nop on x86, but is technically required by the KPI

2133 ↗(On Diff #108915)

ditto

sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c
82

Not sure that we need this comment.

sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.h
3

Likely need to update copyright here. Suggest dropping All Rights reserved, though that's a question for your legal department.

stand/efi/loader/conf.c
90

This seems wrong, and the #define is weird too...
So I'd like more details...
Because we don't build stand with that define, as far as I know, normally.

Device "ttyAMA" refers to a UART with hardware part # PL011.
It's a UART that is specific to ARM processors.
Linux has a driver for this UART at drivers/tty/serial/amba-pl011.c.
In a Hyper-V VM on ARM64, "COM1" refers to a virtual PL011 UART that Hyper-V provides to the guest.
Set-VMFirmware -VMName <vm> -console COM1 adds a ACPI table for SPCR.
Current EFI boot loader comconsole implementation is not supporting ttyAMA but causing a hung during probe.
Until the time FreeBSD boot loader comconsole is supporting ttyAMA, we need to avoid using comconsole for
Hyper-V in ARM64.

I was about to test the match to amd64 Hyper-V but a simple "patch -C < .....diff" wasn't successful.

--------------------------
|Index: sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
|===================================================================
|--- sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
|+++ sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
--------------------------
File to patch:

Is there anything special to get this patch applied?

stand/efi/loader/conf.c
90

The trouble is that we can't control for hypervisor vs non-hypervisor in our build. We build one loader.efi for aarch64, and the non-hypervisor code needs comconsole. It just uses the UEFI Serial interfaces, so in theory should support any device that provides a UEFI driver. It's not the same as the x86 comconsole which talks to hardware directly (yes, this is confusing)

schakrabarti_microsoft.com added inline comments.
stand/efi/loader/conf.c
90

That is why I kept this aarch64_hyperv to avoid comconsole in consoles list, till the time we are getting a ttyAMA support for comconsole
in loader.efi. Once that support is there, we can simply remove it. If you have some other suggestions, please let me know.

sys/arm/conf/GENERIC
285 ↗(On Diff #108915)

No, will remove it. Thanks for pointing.

sys/conf/files.arm64
609

Please add a space after the '#'.

sys/dev/hyperv/vmbus/aarch64/hyperv_aarch64.c
5

This can be dropped.

sys/dev/hyperv/vmbus/aarch64/hyperv_machdep.c
3

This can be dropped.

sys/dev/hyperv/vmbus/aarch64/hyperv_reg.h
3

This can be dropped.

sys/dev/hyperv/vmbus/i386/hyperv_machdep.c
31

Does it mean to be amd64/hyperv_machdep.h or we need a x86/hyperv_machdep.h ?

In D36052#818681, @gbe wrote:

I was about to test the match to amd64 Hyper-V but a simple "patch -C < .....diff" wasn't successful.

--------------------------
|Index: sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
|===================================================================
|--- sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
|+++ sys/dev/hyperv/vmbus/amd64/hyperv_machdep.h
--------------------------
File to patch:

Is there anything special to get this patch applied?

I have tried patch < /mnt/resource/sandbox1/src/patch_arm.diff , and it has worked for me.
I have cloned the repo from github and applied the patch in main branch.

I have removed storvsc changes from here, as it will be reviewed through a separate bug. Also addressed the comments.

Fixed one comment and header inclusion.