Native PCI-express HotPlug support.

[jhb]

Summary

Native PCI-express HotPlug support.

PCI-express HotPlug support is implemented via bits in the slot
registers of the PCI-express capability of the downstream port along
with an interrupt that triggers when bits in the slot status register
change.

This is implemented for FreeBSD by adding HotPlug support to the
PCI-PCI bridge driver which attaches to the virtual PCI-PCI bridges
representing downstream ports on HotPlug slots. The PCI-PCI bridge
driver registers an interrupt handler to receive HotPlug events. It
also uses the slot registers to determine the current HotPlug state
and drive an internal HotPlug state machine. For simplicty of
implementation, the PCI-PCI bridge device detaches and deletes the
child PCI device when a card is removed from a slot and creates and
attaches a PCI child device when a card is inserted into the slot.

The PCI-PCI bridge driver provides a bus_child_present which claims
that child devices are present on HotPlug-capable slots only when a
card is inserted. Rather than requiring a timeout in the RC for
config accesses to not-present children, the pcib_read/write_config
methods fail all requests when a card is not present (or not yet
ready).

These changes include support for various optional HotPlug
capabilities such as a power controller, mechanical latch,
electro-mechanical interlock, indicators, and an attention button.
It also includes support for devices which require waiting for
command completion events before initiating a subsequent HotPlug
command. However, it has only been tested on ExpressCard systems
which support surprise removal and have none of these optional
capabilities.

At the moment this includes some extra debugging printfs for
HotPlug events I have not yet tested. These could be moved under
bootverbose if not removed altogether.

Also, this code is currently always enabled and is always included.
It could be moved under a PCI_HP option similar to PCI_IOV if desired.

The _OSC handling is also not quite right. Ideally the PCI-PCI bridge
driver would pass a request of "can I do HotPlug" up to the ACPI PCI
bridge driver which would only return true if _OSC consents. For now
this just always requests it but doesn't do anything except whine if
_OSC refuses.

Test Plan

• Tested on a Lenovo x220 and T400. The T400 exercised the DLL timer which fires whining about a missed HotPlug interrupt to note that the Data Layer became active without an associated interrupt.

[adrian]

would you mind putting this behind a PCIE_HOTPLUG define? We don't need this for embedded platforms and I'd like to minimise footprint.

Otherwise, looks good! Thanks!

[jhb]

• Move hotplug under a PCI_HP option.

[imp]

sys/dev/pci/pci_pci.c

1095

why 5 seconds?

1610–1620

Why have two different implementations?

sys/dev/pci/pcib_private.h

133–144

ifdef PCI_HP?

[jhb]

sys/dev/pci/pci_pci.c

1095

It's in the spec. The power light is supposed to blink for 5 seconds so that the operator has the chance to push it again to cancel an eject request. If they don't cancel it, then the device is supposed to be detached (powered down, EI released, etc.) after the 5 seconds expires.

1610–1620

pcib_hotplug_present is only available under #ifdef PCI_HP.

sys/dev/pci/pcib_private.h

133–144

I'm a bit leery to have variant-sized softc's since subclass drivers may want to append items to the structure. NEW_PCIB is a bit different in my mind since it should be a platform-wide decision whereas individual kernels may choose to leave out PCI_HP to save space (e.g. embedded systems with stripped-down kernels). However, if a subclass doesn't include "opt_pci.h" it wouldn't get a compile error or warning, just corruption at runtime.

[vangyzen]

sys/dev/pci/pci_pci.c

886

Why not return here?

967

Passing observation: -1 seems like a strange way to indicate truth.

[vangyzen]

sys/dev/pci/pci_pci.c

1197

Should this be "else if"? I don't see a reason to simulate the interrupt if the DLL is inactive.

[vangyzen]

sys/dev/pci/pci_pci.c

1220

What did you intend here? (...by essentially assigning count to itself)

[jhb]

sys/dev/pci/pci_pci.c

886

Oh, that's missing. It should return.

[jhb]

sys/dev/pci/pci_pci.c

967

It is because it is used to implement bus_child_present() which has this semantic. (-1 means device present, 0 means absent, > 0 means error trying to determine and/or not sure)

1197

Hmm, probably, yes. I should perhaps force a detach in this case which will power down the card as well.

1220

Originally I was going to support arbitrary counts but only request 1 and then "move" the 1 interrupt to the right slot. However, for now I've decided to punt until somehow shows up with such a beast (slot with multiple MSI-X messages). The assignment is leftover from the previous code.

[jhb]

• Return if the slot accepts direct hotplug commands.
• Fix issues in DLL Active timeout.
• Remove redundant assignment.

[jhb]

• Stop the DLL timer if the link becomes active.
• Move some debug printfs under bootverbose.