Page MenuHomeFreeBSD

Use PCI power-mgmt to reset a device if FLR fails.
ClosedPublic

Authored by grehan on May 2 2018, 4:57 PM.

Details

Summary

A large number of devices don't support PCIe FLR, in particular
graphics adapters. Use PCI power management to perform the
reset if FLR fails or isn't available, by cycling the device
through the D3 state.

This has been tested by a number of users with Nvidia and AMD GPUs.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 16397
Build 16322: arc lint + arc unit

Event Timeline

grehan created this revision.May 2 2018, 4:57 PM
imp accepted this revision.May 2 2018, 5:01 PM

I click 'ok' with some reservations, however. Really old PCI devices may have issues with this, and the reset by going into D3 state isn't complete on some devices. However, despite those reservations, I think for the set of hardware that's relevant for FreeBSD and given that D3->D0 reset is the best we have for some devices it's a reasonable tradeoff. Since it's not in the bus code itself, it should present no problems.

This revision is now accepted and ready to land.May 2 2018, 5:01 PM
grehan added a comment.EditedMay 2 2018, 5:14 PM

bhyve passthru is currently restricted to the subset of devices that can support MSI/MSI-x so this doesn't come into play for older PCI devices.

rgrimes accepted this revision.May 2 2018, 5:14 PM

Is this the workaround developed with macy?

grehan added a comment.May 2 2018, 5:15 PM

Yes, this code was submitted by Matt M.

jhb accepted this revision.May 2 2018, 5:18 PM

Yes, this is effectively the patch from Matt which I think I've reviewed before. I think this is fine. Some devices also support FLR via non-PCI-e means (PCI advanced features capability) but I don't think the GPUs do.

imp added a comment.May 2 2018, 5:34 PM

bhyve passthru is currently restricted to the subset of devices that can support MSI/MSI-x so this doesn't come into play for older PCI devices.

So much the better. All the rogues I know in this area are pre-PCIe, pre-MSI.

This revision was automatically updated to reflect the committed changes.