Page MenuHomeFreeBSD

bhyve: GPU Passthrough for dedicated AMD GPUs
AbandonedPublic

Authored by corvink on Dec 3 2020, 9:42 AM.
Tags
Referenced Files
Unknown Object (File)
Mon, Mar 18, 2:44 AM
Unknown Object (File)
Jan 28 2024, 12:45 AM
Unknown Object (File)
Jan 27 2024, 10:15 AM
Unknown Object (File)
Jan 27 2024, 10:15 AM
Unknown Object (File)
Jan 27 2024, 10:15 AM
Unknown Object (File)
Jan 24 2024, 11:43 PM
Unknown Object (File)
Jan 23 2024, 12:54 AM
Unknown Object (File)
Jan 14 2024, 3:53 PM

Details

Reviewers
None
Group Reviewers
bhyve
Summary

Description:
This patch adds support for GPU Passthrough of dedicated AMD GPUs. No integrated or second dedicated GPU is neccessary to use GPU Passthrough. If no second GPU is installed, FreeBSD will run headless after booting your VM.

It's based on some other reviews:

To simplify testing, this patch includes all changes. I'm going to rebase this patch as soon as other reviews are merged.

Prerequisites:

  • CPU with VT-d or AMD-Vi

Tested Scenarios:

HardwareOS
Intel Xeon D-1527 + AMD E8860Windows 10Working
Intel Xeon D-1527 + AMD E8860Ubuntu 20.04Working

For Windows it's only neccessary to use another BAR-emulation for passthru devices.
Linux GPU driver is searching for a VBIOS which is currently missing. This patch tries to extract VBIOS and passthrough it as PCI ROM to a guest.

How to use GPU Passthrough:

  1. Use newest UEFI with enabled bus enumeration:
    • get latest edk2 sources
      • portsnap fetch extract
      • cd /usr/ports/sysutils/uefi-edk2-bhyve
      • make extract
    • modify OvmfPkg/Bhyve/BhyveX64.dsc:
      • set gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration to FALSE:

        gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|FALSE
    • rebuild and install new UEFI
      • cd /usr/ports/sysutils/uefi-edk2-bhyve
      • make reinstall
  1. Just add your GPU to bhyve like any other pci passthru device:
    • assign ppt driver to your gpu
    • add your gpu as passthru device to your bhyve config (e.g.: -s 2,passthru,3/0/0)
  1. To use a custom Video-BIOS add the path to the romfile:
    • -s 2,passthru,3/0/0,rom=<path/to/romfile>

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

Are there any technical limitations that make this specific to AMD GPU?

Are there any technical limitations that make this specific to AMD GPU?

This patch extracts VBIOS only for AMD GPUs and passes it to the guest as PCI ROM.
If it's not neccessary for Nvidia GPUs to extract VBIOS and pass it to the guest, it'll work for them too. So far, I don't have a Nvidia GPU to test it.

corvink edited the summary of this revision. (Show Details)
  • Use new UEFI provided by D27230
corvink retitled this revision from [WIP] bhyve: GPU Passthrough for dedicated AMD GPUs to bhyve: GPU Passthrough for dedicated AMD GPUs.Feb 1 2021, 7:48 AM
khng added inline comments.
usr.sbin/bhyve/pci_emul.c
1302โ€“1303
1302โ€“1303

Unused variables.

corvink edited the summary of this revision. (Show Details)
  • Rebase onto main
  • It's possible to load a Video-BIOS from a file now
corvink edited the summary of this revision. (Show Details)
  • rebase

GPU passthrough for deciated AMD GPUs will be enabled by D33129.