Page MenuHomeFreeBSD

Enhance qpi.c to make it usable on all Core-microarchitecture Xeons.
ClosedPublic

Authored by kib on Sep 7 2017, 3:20 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Nov 19, 9:09 AM
Unknown Object (File)
Wed, Nov 13, 5:33 PM
Unknown Object (File)
Sat, Nov 9, 3:37 AM
Unknown Object (File)
Mon, Oct 28, 5:48 AM
Unknown Object (File)
Sep 27 2024, 1:58 PM
Unknown Object (File)
Sep 25 2024, 12:36 PM
Unknown Object (File)
Sep 22 2024, 2:37 PM
Unknown Object (File)
Sep 19 2024, 10:16 AM
Subscribers

Details

Summary

Existing qpi.c makes too many assumptions about the uncore configuration buses location and about slots occupied. Also it restricts itself only to Nehalem CPUs. It should be useful on all Core-based Xeons. On the 2600 v2 (Ivy) machine I have access to, the buses have numbers 31 (BSP socket) and 63 (second socket), and there is no functions pci0.31.0.0 or pci0.63. According to EDS, all devices on the uncore bus occupy slot >= 8.

Scan all buses, not stopping on the first failed match. Scan all slots for function 0 on the found bus, for instance on Ivy there the slot 0 is not decoded at all. Since the scan is quite unsafe, and access to the buses is mostly useful for developers, enable the csr buses scan with the tunable.

Practically, the attach to config buses is required for the intel-pcm pcm-memory.x tool to work, for instance.

I wonder whether qpi attach should have lower priority to ensure that other bus drivers already attached, so that we do not force some innocent normal PCI bus under qpi.

Test Plan

On two-socket Ivy with devinfo I see two pci buses under qpi0, as expected. Also, pciconf -lvc shows the devices which I want to access, for instance

none101@pci0:31:19:4:	class=0x088000 card=0x18a8103c chip=0x0e818086 rev=0x04 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Registers'
    class      = base peripheral
none102@pci0:31:19:5:	class=0x110100 card=0x18a8103c chip=0x0e368086 rev=0x04 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring'
    class      = dasp
    subclass   = performance counters
none103@pci0:31:22:0:	class=0x088000 card=0x18a8103c chip=0x0ec88086 rev=0x04 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Xeon E7 v2/Xeon E5 v2/Core i7 System Address Decoder'
    class      = base peripheral

The pcm-memory.x works on the patched kernel while it refuses to start on unpatched one.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 11414

Event Timeline

Update after whitespace fixes committed.

The reason for the original restrictions (Nehalem only, etc.) was out of an abundance of caution. We should have some good way of ensuring we only do this probe later than ACPI. Right now there isn't an easy way to do that until we use multipass more on x86 (at least that I can think of).

sys/x86/pci/qpi.c
116

This comment should be updated since we now check every possible slot.

120–124

Maybe use PCIR_SLOTMAX (or whatever the constant is called) rather than the bare 0x20?

This revision is now accepted and ready to land.Sep 8 2017, 6:27 PM
kib marked 2 inline comments as done.Sep 8 2017, 7:08 PM
kib added inline comments.
sys/x86/pci/qpi.c
120–124

PCI_SLOTMAX. Then PCI_BUSMAX should be used instead of 255.

kib marked an inline comment as done.

Use PCI_SLOTMAX.
Adjust comment.

This revision now requires review to proceed.Sep 8 2017, 7:09 PM
This revision was automatically updated to reflect the committed changes.