Page MenuHomeFreeBSD

gicv3_its: Don't restrict target CPUs based on SRAT
ClosedPublic

Authored by scottph on Jan 25 2021, 8:18 PM.

Details

Summary

ACPI Sec 5.2.16.5 (SRAT, GIC Interrupt Translation Service (ITS)
Affinity Structure) says:

The GIC ITS Affinity Structure provides the association between
a GIC ITS and a proximity domain. This enables the OSPM to
discover the memory that is closest to the ITS, and use that in
allocating its management tables and command queue.

Previously the ITS driver was using the proximity domain to
restrict which CPUs can be targeted by an LPI. We keep that logic
just for the original dual socket ThunderX which cannot forward
LPIs between sockets.

We also use the SRAT entry for its intended purpose of attempting
to allocate ITS table structures near the ITS.

Sponsored by: Ampere Computing LLC

Test Plan

Boot on Altra with the CPU configured in Hemisphere or Quadrant mode with an nvme device attached to a pcie controller connected to a domain other than the boot domain.

Diff Detail

Repository
R10 FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

We need to limit which CPUs we send these interrupts to on the original dual socket ThunderX as it is unable to forward them between the two sockets.

scottph edited the summary of this revision. (Show Details)

We need to limit which CPUs we send these interrupts to on the original dual socket ThunderX as it is unable to forward them between the two sockets.

Would this be a reasonable approach? It seems like the GIC architecture doesn't give us any help here, other than "the ITS channel to the Redistributors is IMPLEMENTATION DEFINED."

Is the IIDR for the existing quirk the right one to hook for the case you mention? Could there be other implementations with this restriction? I could also set sc_cpus to all_cpus only for Altra if you think it would be better to be conservative.

This revision is now accepted and ready to land.Thu, Feb 4, 3:50 PM
sys/arm64/arm64/gicv3_its.c
249

Most fields here have an sc_ prefix, it'd be nice to try and keep it consistent.

887

Don't you want to allocate these from the domain local to the CPU?

This revision now requires review to proceed.Tue, Feb 16, 6:07 PM
scottph added inline comments.
sys/arm64/arm64/gicv3_its.c
887

Right you are, I had mistaken this for the hardware's collection table which is actually allocated up on line 492.

This revision is now accepted and ready to land.Tue, Feb 16, 6:23 PM
This revision was automatically updated to reflect the committed changes.
scottph marked an inline comment as done.