Page MenuHomeFreeBSD

bhyve: xhci: take the controller lock in the device interrupt path
Needs ReviewPublic

Authored by wanpengqian_gmail.com on Wed, Oct 7, 6:22 AM.

Details

Reviewers
kevans
Group Reviewers
bhyve
Summary

A device's reports (the tablet, a keyboard) are delivered on the back end's
own thread through hci_intr (pci_xhci_dev_intr()). It runs the endpoint and
writes the Event Ring without the controller lock, while vCPU threads insert
command-completion and transfer events and recount the ring on ERDP writes
under sc->mtx. Two writers on one ring lose events or leave a slot with a
stale cycle bit, which stops the guest's handler there, and the event count
drifts until the controller reports Event Ring Full.

Observed on a Windows 10 guest driven over a remote-desktop front end: fast
pointer input made Windows reset the controller again and again (Stop
Endpoint then HCRST) and finally give up, leaving the tablet gone.

Take sc->mtx around the device interrupt, the same lock every MMIO access
already holds; nothing calls hci_intr holding it.

Found and fixed in keelOS (keelos.dev).
Signed-off-by: Wanpeng Qian <wanpengqian@gmail.com>
Sponsored by: keelos.dev

Test Plan

The device interrupt path (pci_xhci_dev_intr, hci_intr) runs on the back end's
input thread and wrote the Event Ring and an endpoint's transfer ring with no
lock, while vCPU threads do so under sc->mtx. A trace counting "another thread
on the event ring" fired during fast input, each time 0-3 ms before the guest
issued Stop Endpoint and HCRST.

Reproduced on a Windows 10 guest (xhci,tablet) over a remote-desktop front end
with a scripted window-drag storm: without the lock Windows reset the
controller repeatedly (19 HCRST in 11 min) and finally dropped the tablet
(yellow mark, no root hub, PS/2 mouse left). With sc->mtx taken in the device
interrupt the storm is gone and the tablet stays. (Verified on real hardware.)

No regression on the nested FreeBSD-main tester: a FreeBSD guest with xhci on
its own pin and on a pin shared with ahci0 boots under INTx and MSI, idle
interrupt rate 0/s, the tablet delivers pointer events.

Found and fixed in keelOS.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 77795
Build 74678: arc lint + arc unit