bhyve: add suport for other mappings in the MSI-X table page
AbandonedPublic
Actions

Authored by bz on Jul 20 2021, 3:19 PM.

Details

Reviewers

grehan
jhb

Group Reviewers

bhyve

Summary

Intel AX200 seems to map other usable address space into the
same page holding the the MSI-X table (which may be against the spec).
Map the host view of MSI-X Table page of the bar and perform
reads/writes on that for anything not covered by the MSI-X PBA or
the MSI-X Table.

Many thanks to grehan and jhb for helping to figure this out.

Diff Detail

Repository

rS FreeBSD src repository - subversion

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 40561
Build 37450: arc lint + arc unit

Event Timeline

bz created this revision.Jul 20 2021, 3:19 PM

Herald added a reviewer: bhyve. · View Herald TranscriptJul 20 2021, 3:19 PM

Herald added subscribers: bcran, rgrimes, imp. · View Herald Transcript

bz requested review of this revision.Jul 20 2021, 3:19 PM

Harbormaster completed remote builds in B40561: Diff 92500.Jul 20 2021, 3:19 PM

There is still an issue with this I believe in case the Table grows larger than 1 page.
now I would hope that we would never see these kinds of devices but I'll leave this and other things to the review.

For others showing the MSI-X Table not in a different bar:

bar   [10] = type Memory, range 64, base 0x9e200000, size 16384, enabled
    cap 11[80] = MSI-X supports 16 messages, enabled
                 Table in map 0x10[0x2000], PBA in map 0x10[0x3000]

and a random read/write from within the same page as the MSI-X Table as seen:

passthru_read: baridx 0 msixtabbar 0 offset 0x2804 size 4 val 0x2affe0 error 0
passthru_write:1071: baridx 0 msixtabbar 0 offset 0x2804 size 4 val 0x28ffe0 error 0

In D31241#703457, @bz wrote:

There is still an issue with this I believe in case the Table grows larger than 1 page.
now I would hope that we would never see these kinds of devices but I'll leave this and other things to the review.

I posted a related review to avoid using /dev/mem for this: D31534. Right now it only maps the single page containing the PBA, but I suppose we could just map the entire BAR and use that...?

For others showing the MSI-X Table not in a different bar:

bar   [10] = type Memory, range 64, base 0x9e200000, size 16384, enabled
    cap 11[80] = MSI-X supports 16 messages, enabled
                 Table in map 0x10[0x2000], PBA in map 0x10[0x3000]

and a random read/write from within the same page as the MSI-X Table as seen:

passthru_read: baridx 0 msixtabbar 0 offset 0x2804 size 4 val 0x2affe0 error 0
passthru_write:1071: baridx 0 msixtabbar 0 offset 0x2804 size 4 val 0x28ffe0 error 0

Any idea what this region is used for?

emaste added a subscriber: emaste.Aug 31 2021, 1:10 AM

markj mentioned this in D32359: bhyve: Map the MSI-X table unconditionally when passthrough is enabled.Oct 7 2021, 7:58 PM

Abandon in favour of D32359