Page MenuHomeFreeBSD

bhyve: GPU Passthrough for dedicated AMD GPUs
Needs ReviewPublic

Authored by c.koehne_beckhoff.com on Dec 3 2020, 9:42 AM.

Details

Reviewers
None
Group Reviewers
bhyve
Summary

Description:
This patch adds support for GPU Passthrough of dedicated AMD GPUs. No integrated or second dedicated GPU is neccessary to use GPU Passthrough. If no second GPU is installed, FreeBSD will run headless after booting your VM.

It's based on some other reviews:

To simplify testing, this patch includes all changes. I'm going to rebase this patch as soon as other reviews are merged.

Prerequisites:

  • CPU with VT-d or AMD-Vi

Tested Scenarios:

HardwareOS
Intel Xeon D-1527 + AMD E8860Windows 10Working
Intel Xeon D-1527 + AMD E8860Ubuntu 20.04Working

For Windows it's only neccessary to use another BAR-emulation for passthru devices.
Linux GPU driver is searching for a VBIOS which is currently missing. This patch tries to extract VBIOS and passthrough it as PCI ROM to a guest.

How to use GPU Passthrough:

  1. Use newest UEFI with enabled bus enumeration:
    • get latest edk2 sources
      • portsnap fetch extract
      • cd /usr/ports/sysutils/uefi-edk2-bhyve
      • make extract
    • modify OvmfPkg/Bhyve/BhyveX64.dsc:
      • set gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration to FALSE:

        gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|FALSE
    • rebuild and install new UEFI
      • cd /usr/ports/sysutils/uefi-edk2-bhyve
      • make reinstall
  1. Just add your GPU to bhyve like any other pci passthru device:
    • assign ppt driver to your gpu
    • add your gpu as passthru device to your bhyve config (e.g.: -s 2,passthru,3/0/0)
  1. To use a custom Video-BIOS add the path to the romfile:
    • -s 2,passthru,3/0/0,rom=<path/to/romfile>

Diff Detail

Repository
R10 FreeBSD src repository
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

Are there any technical limitations that make this specific to AMD GPU?

Are there any technical limitations that make this specific to AMD GPU?

This patch extracts VBIOS only for AMD GPUs and passes it to the guest as PCI ROM.
If it's not neccessary for Nvidia GPUs to extract VBIOS and pass it to the guest, it'll work for them too. So far, I don't have a Nvidia GPU to test it.

c.koehne_beckhoff.com edited the summary of this revision. (Show Details)
  • Use new UEFI provided by D27230
c.koehne_beckhoff.com retitled this revision from [WIP] bhyve: GPU Passthrough for dedicated AMD GPUs to bhyve: GPU Passthrough for dedicated AMD GPUs.Feb 1 2021, 7:48 AM
khng added inline comments.
usr.sbin/bhyve/pci_emul.c
1353–1354
1353–1354

Unused variables.

c.koehne_beckhoff.com edited the summary of this revision. (Show Details)
  • Rebase onto main
  • It's possible to load a Video-BIOS from a file now