acpi: Fix domain ID lookup for grandchild devices

[ziaee]

Summary

During boot on GCE c4a-metal instances, the NVMe driver failed to attach
because bus_get_domain() returned a garbage NUMA domain ID, leading to a
kernel panic during MSI-X allocation.

In the FreeBSD Newbus architecture, only the immediate parent bus knows
the layout of its child's Instance Variables (IVARs). acpi0 knows the
layout of struct acpi_device for its direct children, while pci0 knows
struct pci_devinfo for PCI devices.

When bus_get_domain() is called on a grandchild PCI device (e.g. nvme0
under pci1 -> pcib1 -> pci0), the request delegates up to acpi0.
However, acpi_get_domain_method() incorrectly passed the original
grandchild device_t to acpi_read_ivar(), casting a struct pci_devinfo
pointer to struct acpi_device. This type confusion read garbage data
from the PCI device info structure as the domain ID.

Fix this by walking up the device tree in acpi_get_domain_method() to
identify the ancestor that is a direct child of acpi0 (the PCI Host
Bridge, e.g. pcib0). The ACPI domain IVAR is then safely read from this
direct child. This pattern matches existing ancestor-walking logic in
acpi_get_cpus().

Note: The SRAT, DSDT, and APIC dumps are on freebsd.org/~ziaee/tmp/c4a-*

Authored by: Jasper Tran O'Leary <[e-mail]>
Sponsored by: Google Cloud

[ziaee]

ping

[jhb]

Hmmm, I need to think on this a bit. We tend to not do these ancestor works in methods. Instead, for cases like this we often fix the buses below to change which device is passed up as the child instead. In particular, see acpi_pcib_get_cpus used by both the ACPI-aware Host-PCI and PCI-PCI bridge drivers. This translates a bus_get_cpus request for a child device into a request for the bridge device. I think the similar approach is probably the better model to follow here. This method should perhaps bail with ENXIO or the like if child is not a direct child of dev as extra protection though. I'm not aware of any other bridge-type drivers that are children of ACPI other than the PCI bridge drivers that would need this fix. ARM insists on using their own PCI host bridge drivers which might need updating (the poorly named pci_host_generic.c). (Those also need to use acpi_pcib_get_cpus actually)