**Description:**
Add support of GPU Passthrough for integrated graphics of AMD Ryzen V1000 APUs.
//Note://
This patch is currently not working on any system.
This patch is based on [[ https://reviews.freebsd.org/D26209 | D26209 ]].
**Prerequisites:**
- AMD Ryzen V1000 APU with Vega graphics
**How to enable GPU Passthrough:**
AddUse the new "apu" option to the-d" device for your graphic devices card.
Your bhyve config could look like:
```
bhyveload -S -m 4G -d /root/fbsd/fbsd.img fbsd
bhyve -c 41 -m 4G -A -H -P -S \-HwS \
-s 0,hostbridge \
-s 2:0,passthru,apu-d,4/0/0,apu \ \
-s 5,virtio-blk-s 4,ahci-cd,/root/fbsd/fbsd.img \iso \
-s 5,ahci-hd,/root/fbsd/fbsd.img \
-s 6,virtio-net,tap10 \
-s 301,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd \
fbsd
```
**Tested Scenarios:**
CPU: AMD Ryzen Embedded V1807B
| **OS** | |
| Windows | Not Working |
| Debian | Not Working |
| FreeBSD | Not Working |
**Installation Steps:**
1. Apply patch [[ https://reviews.freebsd.org/D26209 | D26209 (Diff 4) ]] and this patch to your FreeBSD sources
2. Rebuild kernel and bhyve
3. Install a FreeBSD VMApply sysutils_uefi-edk2-bhyve_files_2019-07-19.tgz patches to /usr/ports/sysutils/uefi-edk2-bhyve-csm
4. Log in to your Guest-OS:Rebuild /usr/ports/sysutils/uefi-edk2-bhyve-csm
15. Apply patch [[ https://reviews.freebsd.org/D26209 | D26209 (Diff 4) ]] and this patch to your FreeBSD sourcesStart vm with following config:
2. Rebuild kernel```
3. Install drm-kmod (`pkg install drm-kmod`)bhyve -c 1 -m 4G -HwS \
4. Build and install drm-kmod from source -s 0,hostbridge \
- `git clone https://github.com/freebsd/drm-kmod`-s 2,apu-d,4/0/0 \
- `cd drm-kmod`-s 4,ahci-cd,/root/fbsd/fbsd.iso \
- `make && make install`-s 5,ahci-hd,/root/fbsd/fbsd.img \
5. Load amdgpu driver -s 6,virtio-net,tap10 \
- `kldload /boot/modules/linuxkpi_gplv2.ko`-s 31,lpc \
- `kldload /boot/modules/drm.ko`-l com1,stdio \
- `kldload /boot/modules/ttm.ko`-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd \
- `kldload /boot/modules/amdgpu.ko`fbsd
```
**Known Limitations:**
- Detection of VBIOS address only uses VFCT table. `drm-kmod/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c` contains more options
**Known Issues:****What I've done so far:**
- Load of amdgpu driver fails:1. Extract PCI ROM support from [[ https://www.gulbra.net/freebsd-bhyve/ | vga-bios patches ]]
Following output is displayed on `kldload /boot/modules/amdgpu.ko`:2. Add an ioctl to get VBIOS from VFCT
3. Add an apu-d device
- It's a passthru device
- It maps VBIOS into PCI ROM
**Known Issues:**
1. EFI fails with following message:
```
drmn0: successfully loaded firmware image with name: amdgpu/raven_vcn.binLegacy Boot: CDROM 0
<6>[drm] Found VCN firmware Version ENC: 1.9 DEC: 1 VEP: 0 Revision: 28Start of legacy boot
<6>[drm] PSP loading VCN firmwareE820[ 0]: 0x 0 ---- 0x 9FC00, Type = 0x1
<6>[drm] reserve 0x400000 from 0xf43f80E820[ 1]: 0x 9FC00 ---- 0x A0000 for PSP TMR, Type = 0x2
<4>[drm] psp command failed and response status is (0xFFFF)E820[ 2]: 0x E0000 ---- 0x 100000, Type = 0x2
[drm ERROR :psp_hw_start] PSP load tmr failed!E820[ 3]: 0x 100000 ---- 0x BBFBE000, Type = 0x1
[drm ERROR :psp_hw_init] PSP firmware loading failedE820[ 4]: 0x BBFBE000 ---- 0x BBFDE000, Type = 0x3
[drm ERROR :amdgpu_device_fw_loading] hw_init of IP block <psp> failed -22E820[ 5]: 0x BBFDE000 ---- 0x BE1D8000, Type = 0x1
drmn0: amdgpu_device_ip_init failedE820[ 6]: 0x BE1D8000 ---- 0x BE884000, Type = 0x3
drmn0: Fatal error during GPU initE820[ 7]: 0x BE884000 ---- 0x BE8B4000, Type = 0x1
E820[ 8]: 0x BE8B4000 ---- 0x BF784000, Type = 0x3
E820[ 9]: 0x BF784000 ---- 0x BF789000, Type = 0x1
E820[10]: 0x BF789000 ---- 0x BF958000, Type = 0x3
E820[11]: 0x BF958000 ---- 0x BF95C000, Type = 0x2
E820[12]: 0x BF95C000 ---- 0x BF964000, Type = 0x3
E820[13]: 0x BF964000 ---- 0x BF968000, Type = 0x4
E820[14]: 0x BF968000 ---- 0x BFFF0000, Type = 0x3
E820[15]: 0x BFFF0000 ---- 0x C0000000, Type = 0x1
E820[16]: 0x C0000000 ---- 0x FC000000, Type = 0x2
E820[17]: 0x FEC00000 ---- 0x FEC01000, Type = 0x2
E820[18]: 0x FED00000 ---- 0x FED00400, Type = 0x2
E820[19]: 0x FEE00000 ---- 0x FEF00000, Type = 0x2
E820[20]: 0x 100000000 ---- 0x 140000000, Type = 0x1
PIRQ table in legacy region - F95B0
PCI VGA: 0x1002:0x15DD
PCI VGA: 0x1002:0x15DD
ASSERT /usr/ports/sysutils/uefi-edk2-bhyve-csm/work/uefi-edk2-aa8d718/IntelFrameworkModulePkg/Csm/LegacyBiosDxe/LegacyPci.c(1732): (Status == 0) || (Status == ((RETURN_STATUS)(0x8000000000000000ULL | (14))))
```
If `amdgpu_fw_load_type` (drm-kmod/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c) is set to `0` (AMDGPU_FW_LOAD_DIRECT) instead of `-1`, it'll fail with following messageThis error is caused by `LegacyBiosInstallVgaRom` at line 2211:
```
drmn0: successfully loaded firmware image with name: amdgpu/raven_vcn.bin2204 if (Status == EFI_SUCCESS) {
<6>[drm] Found VCN firmware Version ENC: 1.9 DEC: 1 VEP: 0 Revision: 282205 Private->VgaInstalled = TRUE;
drmn0: [drm:*ERROR* ] amdgpu_ring_test_helper[drm ERROR :gfx_v9_0_kiq_kcq_enable] KCQ enable failed2206
[drm ERROR :amdgpu_device_ip_hw_init_phase2] hw_init of IP block <gfx_v9_0> failed -602207 //
drmn0: amdgpu_device_ip_init failed2208 // Attach the VGA thunk driver.
drmn0: Fatal error during GPU init2209 // Assume the video is installed. This prevents potential of infinite recursion.
2210 //
2211 Status = gBS->ConnectController (
2212 VgaHandle,
2213 ConnectHandleBuffer,
2214 NULL,
2215 TRUE
2216 );
2217 }
```
2. It's possible to press a key on startup to enter Boot Menu. If I select `EFI DVD/CDROM` from `Boot Manager`, FreeBSD will start booting and fails with following message:
```
Unhandled ps2 mouse command 0xe1
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Generic PS/2 mouse, device ID 0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
Unhandled memory access to 0xb0000
Failed to emulate instruction sequence [ f2a4eb110f1f4000488d7dd589c2fc ] at 0xffffffff8120c2e8
Abort trap
```