Page MenuHomeFreeBSD

Add loader support for xhci debug
Needs ReviewPublic

Authored by thj on Oct 30 2025, 1:02 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Mar 29, 3:04 PM
Unknown Object (File)
Sun, Mar 29, 12:40 PM
Unknown Object (File)
Fri, Mar 27, 4:55 PM
Unknown Object (File)
Wed, Mar 25, 2:11 AM
Unknown Object (File)
Mon, Mar 23, 4:15 AM
Unknown Object (File)
Mon, Mar 23, 4:15 AM
Unknown Object (File)
Mon, Mar 23, 2:19 AM
Unknown Object (File)
Tue, Mar 17, 1:04 PM

Details

Summary

This patch implements loader support for xhci debug and carries both the shared
code for both loader and the kernel.

I think there is still probably stuff to do in here, but I would really like
input from some other people on the overall structure and how to proceed.

Description

xhci debug requires an efi implementation with a working pcie bus and
implements the minimum of xhci required to interface with the controller and to
put a port into debug mode.

xhci debug functions as a console in loader and can be used from cons_probe
(quite early in loader/main.c:main). The cable detection process takes a long
time (on the order of a second) to run making enabling xhci debug by default an
impossible proposition.

It is possible to append "udbc" to the static console list in
efi/loader/main.c:main and a future patch will add this as a build option.
There are outstanding bugs with doing this.

Theory of Operation

udbc probes with the normal console list and allocates and creates its required
data structures during its probe/attach process.

Once probed udbc can be used as an input and output console from the C efi and
loader.

The data structures allocated for udbc are kept in a softc which is kept in
sync with the kernel. During boot the softc is bundled up and passed to the
kernel as a machine dependent module (MODINFOMD_XHCI_DEBUG).

The kernel picks up this data in parse_preload_data.

xhci debug is functional once the kernel has picked up the data structures, but
we are lacking an early pci controller to be able to do the required pcie
transactions until the xhci bus probes much later.

When used with a supported kernel, udbc is able to provide a console for
loader, with a gap between kernel start and xhci probe. xhci debug should
continue to be usable until the xhci controller is reset or the pcie bus is
reset.

Other platforms do not support its use during system or bus suspend - it isn't
clear if this is from experience or from attempts to make the problem space
smaller.

When the console wants to send data it adds the output bytes to a work_queue
and then uses the xhci trbs to submit this to the bus.

Periodically (on output and when polled by the console system) usb transfers
are driven and any input bytes are pulled from a work queue as rx data.

Usage

target: Build loader with support and deploy/install it

Connect your debug target to your debug host via a xhci debug cable (a usb3
cable with vbus removed on one end).

host: On your debug host load the udbc kernel module (freebsd 15 or later).

target: From loader menu break to a command prompt and enter

set console="udbc"
or
set console="comconsole udbc"

host: once the cable/target is detected you will have a functioning usb serial

console device

Known issues/caveats

  • there is a stall during loader start up which leads to a single screen of loader menu being rendered then no more if configured as a default console in efi/loader/main.c:main or loader.conf
  • usb-c cables require an adapter to enable debug adapter mode (I'm waiting for pcbs to verify this)
  • the "dead time" when udbc isn't available in the kernel is at the most annoying time.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 70731
Build 67614: arc lint + arc unit

Event Timeline

thj requested review of this revision.Oct 30 2025, 1:02 PM
stand/efi/loader/xhci_dbc_dma.c
283

Should this be dma?

sys/dev/usb/controller/xhci_dbc.c
1055

where is label defined?

stand/efi/loader/xhci_dbc_cons.c
212

This should technically set errno = 0 before and check it after for overflow. Same below.

232

I'd feel a little better if the outer block here had braces

stand/efi/loader/xhci_dbc_dma.c
72

This kinda feels like it should be going through uintptr_t instead (along with the other UINTN usage here

83

Consider asserting the condition instead, since there shouldn't be any other path leading to this that leaves us with a non-NULL.

93
127

howmany?

131
sys/dev/usb/controller/xhci_dbc.c
644

This is presumably synchronizing with trb_push_locked? I think the barrier application is questionable at best, and this should probably be using proper atomics on both sides...

722

Ditto above note about atomics

thj marked 7 inline comments as done.Feb 11 2026, 10:49 AM
kevans added inline comments.
sys/dev/usb/controller/xhci_dbc.c
644

More specifically, I'm pretty sure all of these loads from trb->dwTrb3 with rmb throughout should be an appropriate atomic(9) load_acq, and trb_push_locked should be rearranged such that the last write to the trb should be a store_rel to dwTrb3. This does mean getting rid of a cleaner memset, but I think the more narrow barrier is worth it. CC @kib, maybe

stand/efi/loader/bootinfo.c
464

Do you intent to pass pointer to the struct, or the struct itself? If a pointer, what makes the memory allocation survive to the kernel runtime?

stand/efi/loader/xhci_dbc_cons.h
29

Do we need include braces for double-inclusion protection?

Same question for all added headers.

sys/dev/usb/controller/xhci_dbc.c
31

The sys/systm.h should go first, then sys/param.h is not needed

36

We have _STANDALONE I remember. IMO it is (much) better to write #ifdef _STANDALONE than #ifndef _KERNEL.

644

What is the semantic needed from this barrier?

653

And from this one?

Hope these are helpful, lemme know if you have any questions / concerns. Thanks!

stand/efi/loader/bootinfo.c
73

Can you put this into a new file? bootinfo.c doesn't need this infection... It's shared with kboot, and I'd prefer we keep niche things like this out of there.

Maybe xhci_dbc_meta.c is a better place to implement the usbdbc_add_metadata routine I suggest below? It doesn't fit well with the other files, but it's also small so could fit into xhci_dbc_cons.c.

464

I'd prefer this be something like usbdbc_add_metaddata(kfp); and this code be elsewhere.

stand/efi/loader/xhci_dbc_cons.c
75

where's device_t defined? It looks like we're reading it from the PCI device, so that has me scratching my head... It's just the device id, so device_t seems like an odd choice. There doesn't seem any reason to make this "device_t" since all the definitions are self contained.

112

silly style nit: No space after (void).
But also, why have it return bool if we're just going to ignore it?

179

same comment

183

Please fix this to free it.

stand/efi/loader/xhci_dbc_pci.h
69

Yea, I'm not comfortable with this type punning for such a small gain below. There's no real need that I can see for doing this, apart from the one shared bit of code that can be done in a different manner.

sys/dev/usb/controller/xhci_dbc.c
36

I made all my comments about this before I Saw this coment.

56

This is unused and is the only use of device_t in this file.

120

In new code, we don't need (or really want) blank lines like this.

149

_STANDALONE

sys/dev/usb/controller/xhci_dbc.h
31

We usually spell this #ifdef _STANDALONE rather than !KERNEL.

144

same comment. Avoid !defined(KERNEL) constructs please.

202

I'd have two ifdefs. One for the kernel. and then one for _STANDALONE. The code will fail to build when one or the other isn't defined, which is what we want. In the loader, it's means someone messed up CFLAGS. and if we're not in the loader or in the kernel, we want to fail because we've not built out that other environment.

sys/dev/usb/controller/xhci_dbc_private.h
31

#if defined(KERNEL) is the right construct

45

Same applies here for earlier _STANDALONE comments.

sys/dev/usb/controller/xhci_pci.h
30

here, and elsewhere, sys/cdefs.h is almost certainly not needed.

43

See comment in another review, but make this take device_id, class, etc so we don't need device_t to seep into the loader. When I commented on the other review, I didn't notice the need to do the latter. Make the driver and the loader have a wrapper that extracts this information and calls this.

thj marked 13 inline comments as done.

Address review comments

OK, my SIIG cable arrived, and I have yet to test with it. I'll plug in via a hub for safety at first, and to test with udbc(4) as the client end (I used to use udbp(4) for this way back when), I'll need to pass through a USB device in KVM to my FreeBSD VMs, which at libvirt level requires an XML edit and restart.

I can point the EFI MonotonicCounter GSoC volunteer here for hints, I did not see how to pass things from loader to kernel in a more structured way being one of the older BIOS generation folk.

Any hints about how to expedite a loader build with this in the tree? It would require building a USB image?

In D53472#1281146, @bms wrote:

I can point the EFI MonotonicCounter GSoC volunteer here for hints, I did not see how to pass things from loader to kernel in a more structured way being one of the older BIOS generation folk.

file_addmetadata(kfp, MODINFOMD_FW_HANDLE, sizeof(ST), &ST);

Is how it's done. You define a new MODINFOMD_xxxx and you pass it from the loader... The kernel reads with

vm_paddr_t efi_systbl_phys = MD_FETCH(preload_kmdp, MODINFOMD_FW_HANDLE,
    vm_paddr_t);

Reading this review to get those details would be kinda rough... We basically make a note on the kernel to pass this metadata. modulep is one of the args we pass from the loader to the kernel which we then walk to get all this data. so kfp referrs to the kernel, though there's dozens of examples on how to get it to pass the context from the unique counter into the kernel.

In D53472#1281160, @imp wrote:
In D53472#1281146, @bms wrote:

I can point the EFI MonotonicCounter GSoC volunteer here for hints, I did not see how to pass things from loader to kernel in a more structured way being one of the older BIOS generation folk.

file_addmetadata(kfp, MODINFOMD_FW_HANDLE, sizeof(ST), &ST);

Is how it's done. You define a new MODINFOMD_xxxx and you pass it from the loader... The kernel reads with

e.g. MODINFOMD_EFI_MONOTONICCOUNT or something shorter could get defined in
<sys/x86/include/metadata.h> for amd64 and i386 EFI support.

Searching shows MODINFOMD_EFI_MAP is defined differently for arm64 and riscv, so the MODINFO defines are obviously in their own namespace per-architecture (hence the MODINFO*MD* for Machine Dependent naming in the define). This is new to me as well!

vm_paddr_t efi_systbl_phys = MD_FETCH(preload_kmdp, MODINFOMD_FW_HANDLE,
    vm_paddr_t);

Reading this review to get those details would be kinda rough... We basically make a note on the kernel to pass this metadata. modulep is one of the args we pass from the loader to the kernel which we then walk to get all this data. so kfp referrs to the kernel, though there's dozens of examples on how to get it to pass the context from the unique counter into the kernel.

Roger. I pointed them at the file_addmetadata() call already, but didn't know about MD_FETCH().
I did however add a reminder to update sys/kern/subr_module.c to print the values.

In D53472#1281146, @bms wrote:

Any hints about how to expedite a loader build with this in the tree? It would require building a USB image?

That's my main question. I've been busy on business development notes and catching up with IETF intellectual property claim process. And how likely SoftBank would be to litigate if they took a disliking to little old me. But I am not pointy-haired enough yet not to look at Makefiles for loader.efi...

I just don't know for sure if I have DbC capability in my Dell Optiplexes right now, but I noticed the nub in my Acer and ThinkPad's respective xHCI controller info (working from Windows tools, but I've since blown Microsoft away completely on the ThinkPad where my FreeBSD VMs live).

That's why I might need to build a boot image. I can install whatever I like on the Optiplexes, but the laptops are in (non-FreeBSD) production use.

TL;DR Not functional under KVM passthrough of either ThinkPad P14s xHCI node.

The P14S has *two* xHCI controllers: the Thunderbolt one and a secondary one.

NOTE: Need to pass the entire PCI xhci controller device through to a VM.

bms@gun:~$ find /sys -name dbc
...
/sys/devices/pci0000:00/0000:00:0d.0/dbc
/sys/devices/pci0000:00/0000:00:14.0/dbc
...

So pass through /sys/devices/pci0000:00/0000:00:0d.0.
The xHCI function was clearly passed through to gap but on a different
[virtual] bus by QEMU/KVM:

pciconf -lv extract:


xhci0@pci0:2:2:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x8086 device=0x7ec0 subvendor=0x17aa subdevice=0x50e9


xhci0: <XHCI (generic) USB 3.0 controller> mem 0x88a00000-0x88a0ffff at device 2.0 on pci2
xhci0: 32 bytes context size, 64-bit DMA
xhci0: xECP capabilities <PROTO,PROTO,VEND(c0),LEGACY,VEND(c6),VEND(c7),VEND(c2),DEBUG,VEND(c3),VEND(d1),VEND(ce),VEND(c8),VEND(c9),VEND(ca),VEND(cc),VEND(cd),VEND(d2),VEND(cf),VEND(d3)>
usbus0 on xhci0

The two physical USB3 ports on the P14S are being routed to the "secondary"
USB controller, NOT the TB/USB4 one.

Was initially looking at the wrong VM (pin) but tried again (gap) with
physically plugging and replugging, and also using "sudo usbconfig reset
ugen0.1" from gap to force bus re-enumeration. Nothing detected.

Tried with a USB2 wired C-C cable first. thj implied some sort of PCB adapter
might have been needed. Didn't see anything either end.

Tried with a USB4/TB3 wired C-C cable. Didn't see anything either end.
But this was both on the SECONDARY USB-C port, not the one we're taking
power from. Tried swapping them.

Tried the USB 3 "A" ports, which all appear to be mapped to the
same controller bus as the mouse.

Obtain KVM IOMMU group first: virsh nodedev-dumpxml pci_0000_00_14_0

May lose Wi-fi while doing this because:
00:14.0 USB controller: Intel Corporation Meteor Lake-P USB 3.2 Gen 2x1 xHCI Host Controller (rev 20)
00:14.2 RAM memory: Intel Corporation Device 7e7f (rev 20)
00:14.3 Network controller: Intel Corporation Meteor Lake PCH CNVi WiFi (rev 20)

So be prepared to use:
virsh nodedev-reattach pci_0000_00_14_0
virsh nodedev-reattach pci_0000_00_14_2
virsh nodedev-reattach pci_0000_00_14_3

Start out by doing it remotely:
virsh nodedev-dettach pci_0000_00_14_0
virsh nodedev-dettach pci_0000_00_14_2
virsh nodedev-dettach pci_0000_00_14_3

virsh edit gap, and add:


<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/>
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x0d' function='0x2'/>
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x0d' function='0x3'/>
</source>
</hostdev>

The passed-through xhci0 instance reports DEBUG capability in xECP just fine:


xhci0: <XHCI (generic) USB 3.0 controller> mem 0x88a80000-0x88a8ffff at device 2.0 on pci2
xhci0: 32 bytes context size, 64-bit DMA
xhci0: xECP capabilities <PROTO,PROTO,VEND(c0),LEGACY,VEND(c6),VEND(c7),VEND(c2),DEBUG,VEND(c3),VEND(d1),VEND(ce),VEND(c8),VEND(c9),VEND(ca),VEND(cc),VEND(cd),VEND(d2),VEND(cf),VEND(d3)>
usbus0 on xhci0

pciconf -lv extract from SUT:


xhci0@pci0:2:2:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x8086 device=0x7ec0 subvendor=0x17aa subdevice=0x50e9
vendor = 'Intel Corporation'
device = 'Meteor Lake-P Thunderbolt 4 USB Controller'
class = serial bus
subclass = USB
none0@pci0:2:3:0: class=0x0c0340 rev=0x02 hdr=0x00 vendor=0x8086 device=0x7ec2 subvendor=0x17aa subdevice=0x50e9
vendor = 'Intel Corporation'
device = 'Meteor Lake-P Thunderbolt 4 NHI'
class = serial bus
subclass = USB
none1@pci0:2:4:0: class=0x0c0340 rev=0x02 hdr=0x00 vendor=0x8086 device=0x7ec3 subvendor=0x17aa subdevice=0x50e9
vendor = 'Intel Corporation'
device = 'Meteor Lake-P Thunderbolt 4 NHI'
class = serial bus
subclass = USB

Checked if the interposed USB3 hub was detected. not seen on P14S's primary
USB3 port on the left, even with a reset. Same result with the right-hand port.
Tried connecting both USB3 ports via same hub with the SIIG A-A cable.
Nothing detected at all. Tried restarting the VM with this setup. Nothing.
Would rather not risk P14S ports without the hub protectively in place.

Done testing for now.