Details

Reviewers

ziaee
aokblast

Group Reviewers

bhyve
manpages

Summary

This patch introduces userspace-only USB passthrough using libusb. It
supports configuring passthrough via a (passthru.vid.pid) tuple.
Currently, only vendor and product ID-based selection is implemented;
selecting specific ugen devices (to differentiate between identical
devices) would require changes to libusb.

All USB transfer types—control, bulk, interrupt, and isochronous—are
supported. Hotplugging is also handled, allowing devices to be unplugged
when guest startup and replugged afterward without issue.

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 73821
Build 70704: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Fix primary stream id failed

Harbormaster completed remote builds in B66817: Diff 161514.Sep 4 2025, 2:01 PM

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

No difference. The report: https://gist.github.com/kenrap/9a1c406d097aa12c7a7b7ebe106cb681

Release max number of Pstreams

Harbormaster completed remote builds in B66818: Diff 161520.Sep 4 2025, 3:34 PM

In D52166#1196057, @kenrap_kennethraplee.com wrote:

In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

In D52166#1196089, @aokblast wrote:

In D52166#1196057, @kenrap_kennethraplee.com wrote:

In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

So, there is good news and bad news.

Good news is, the boot time went down significantly to 6 seconds.

The bad news, the device is still not detected by the guest OS.

Another report: https://gist.github.com/kenrap/2ad9fc1a216e0c6f7eb089c4f3b44c5f

In D52166#1196117, @kenrap_kennethraplee.com wrote:

In D52166#1196089, @aokblast wrote:

In D52166#1196057, @kenrap_kennethraplee.com wrote:

In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

So, there is good news and bad news.

Good news is, the boot time went down significantly to 6 seconds.

The bad news, the device is still not detected by the guest OS.

Another report: https://gist.github.com/kenrap/2ad9fc1a216e0c6f7eb089c4f3b44c5f

Hello: Does your device work on FreeBSD host? I saw a transfer is cancelled so the problem maybe on the libusb side or kernel stack.

In D52166#1196153, @aokblast wrote:

Hello: Does your device work on FreeBSD host? I saw a transfer is cancelled so the problem maybe on the libusb side or kernel stack.

Sorry for the late reply. Yes, I'm able to mount the device on my FreeBSD host.

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hi, I've tested this with a licensing dongle and a smart card reader on an external USB HUB with Windows 11. Licensing worked, smart card was detected correctly. I am still wondering if param1/2 in the config are the best thing but they work, so be it. Many thanks to you and the FreeBSD Foundation! I'll keep trying and using it for other bits as well the next days on other machines. I just need to disable the debugging as my remote window kept scrolling (have the tablet on for vnc as well in this case) like I was on a 9600 connection downloading Netscape ;-)

avg added a subscriber: avg.Nov 11 2025, 3:13 PM

olevole_olevole.ru added a subscriber: olevole_olevole.ru.Nov 21 2025, 12:38 PM

It's gotten silent here. May I ask what the plan is? Is there a chance to get this work into 16?

In D52166#1232521, @bz wrote:

It's gotten silent here. May I ask what the plan is? Is there a chance to get this work into 16?

Hello. I received the notice of conscription for 4 month obligatory military service in my country in late Octorber which have started from November. I will continue to work on this after my service.

Rebase to main

Harbormaster completed remote builds in B71181: Diff 173134.Mar 4 2026, 3:31 PM

In D52166#1196957, @kenrap_kennethraplee.com wrote:

In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

In D52166#1273524, @aokblast wrote:

In D52166#1196957, @kenrap_kennethraplee.com wrote:

In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

Welcome back! 👋

And thanks for rebasing too. I upgraded my base for the updated patch and others as well.

Sadly the non-detection problem still persists...

I got question though. Is there a way to passthru with a specific ugen number of the same vendor and product IDs?

I'm using multiple of these adapters and they have the same IDs. Only the earliest ugen device gets passthru'd when I need a later one instead.

I've been out of the loop with this so I'm trying to recall what I've forgotten. 🙂

In D52166#1273749, @kenrap_kennethraplee.com wrote:

In D52166#1273524, @aokblast wrote:

In D52166#1196957, @kenrap_kennethraplee.com wrote:

In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

Welcome back! 👋

And thanks for rebasing too. I upgraded my base for the updated patch and others as well.

Sadly the non-detection problem still persists...

I got question though. Is there a way to passthru with a specific ugen number of the same vendor and product IDs?

I'm using multiple of these adapters and they have the same IDs. Only the earliest ugen device gets passthru'd when I need a later one instead.

I've been out of the loop with this so I'm trying to recall what I've forgotten. 🙂

Sure, it is possible by just specify ugen number. I can work on this.
And I need some time to take a look at what I was doing 4 months ago:)

Woohoo. Happy to see updates here.

I've rebased my main once in between; while my smart card reader still works on Win11 after passthru one of my licensing dongles no longer does. I'll grab an updated stack from here the next days and rebuild to see. I haven't had time to debug it anyway so can as well do it on the latest revision. Thanks for all this work!

2khramtsov_gmail.com added a subscriber: 2khramtsov_gmail.com.Mar 6 2026, 2:58 AM

o_fornost.net added a subscriber: o_fornost.net.Mar 17 2026, 5:58 PM

had-churn-float_duck.com added a subscriber: had-churn-float_duck.com.Apr 20 2026, 11:09 AM

In D52166#1273856, @bz wrote:

Woohoo. Happy to see updates here.

I've rebased my main once in between; while my smart card reader still works on Win11 after passthru one of my licensing dongles no longer does. I'll grab an updated stack from here the next days and rebuild to see. I haven't had time to debug it anyway so can as well do it on the latest revision. Thanks for all this work!

Hi, I found a very subtle fails in our xHCI. Please also apply this patch while you are testing.

https://reviews.freebsd.org/D57146

emaste added a subscriber: emaste.Jun 8 2026, 4:40 PM

Large fixes

Harbormaster completed remote builds in B73821: Diff 179604.Jun 11 2026, 3:00 PM

I did a very large regions of bug fixes.

The full patchset is in here https://github.com/aokblast/freebsd-src/tree/bhyve/impl_usbdev_passthru. It should fix many unworking devices.

aokblast edited the test plan for this revision. (Show Details)Jun 11 2026, 3:10 PM

in general, nice work!

usr.sbin/bhyve/usb_passthru.c
762	is err >= 8 here validating the payload length?

aokblast added inline comments.Jun 11 2026, 4:23 PM

usr.sbin/bhyve/usb_passthru.c
762	Yes, see document in https://libusb.sourceforge.io/api-1.0/group__libusb__syncio.html. I might need to rename the variable name.

please also add to manpage in this commit

Add doc

Harbormaster completed remote builds in B73841: Diff 179653.Jun 12 2026, 4:12 PM

Thanks! note that patch does not apply to main

usr.sbin/bhyve/bhyve.8
1070 ↗	(On Diff #179653)	note that the existing Ar tablet is also wrong, tablet is not a variable argument, its a command modifier.
1075 ↗	(On Diff #179653)

Hey, any further traction on this? What's left before it can be landed?

zldrobit_gmail.com added a subscriber: zldrobit_gmail.com.Jul 3 2026, 9:06 AM

bnovkov added a subscriber: bnovkov.Jul 7 2026, 8:05 PM

bnovkov added inline comments.

usr.sbin/bhyve/pci_xhci.c
2977–2986
usr.sbin/bhyve/usb_passthru.c
416–421	Broken formatting?
503	Should you defer setting this until after you've checked the error?
581	We should check the return value here and for `pthread_mutexattr_init` and destroy the mutex in `failed:` only if it was initialized.
636	`bhyve/usb_mouse.c` defines the same macro, would it make sense to split it out into a common header?
814–815	Saving a pointer to `sc->endpoint_types[epctx << 1 \| dir]` would make more sense given that you repeat a non-trivial calculation three times in this function.
930–931

Reflect bnovkov's review

Thanks for your review @bnovkov

usr.sbin/bhyve/usb_passthru.c
636	Is it worth to use another header for just this macro?

Harbormaster completed remote builds in B74742: Diff 181820.Jul 12 2026, 7:14 AM

In D52166#1330369, @adrian wrote:

Hey, any further traction on this? What's left before it can be landed?

There are three things blocking this patch.

We need to upstream the libusb capsicum option enum to upstream as we don't want to break compatibility with upstream
Pass a isoc xfer based device doesn't work (like UVC)
Pass a Hub doesn't work as we can only see the hub device but not its childrens.

I wouldn't over-index on trying to get everything perfect and upstreamed before landing stuff here.

capsicum is a freebsd-ism here; upstreaming it is going to require a bunch of #ifdef entries which D51865 doesn't have.
you can just say that ISO transfers aren't yet supported in passthrough.
you can also just say that hub child device enumeration isn't yet supported in passthrough.

If it's useful now to a bunch of people I think we should focus on what it currently supports and get it landed.
I'd hate to see this languish here because it's not perfect. :-)

cracauer added a subscriber: cracauer.Jul 12 2026, 5:26 PM

In D52166#1334510, @adrian wrote:

If it's useful now to a bunch of people I think we should focus on what it currently supports and get it landed.
I'd hate to see this languish here because it's not perfect. :-)

Is there any way to assist from the outside? Is anyone willing to prepare a proposal to the Foundation to wrap this up?

In D52166#1356953, @editor_callfortesting.org wrote:

In D52166#1334510, @adrian wrote:

If it's useful now to a bunch of people I think we should focus on what it currently supports and get it landed.
I'd hate to see this languish here because it's not perfect. :-)

Is there any way to assist from the outside? Is anyone willing to prepare a proposal to the Foundation to wrap this up?

Kyle E kindly updated us on a Production User Call about progress on the dependencies for this. Thank you!

We all look forward to the ability to pass through our eID cards, software license dongles, Ubikeys, webcams, HDMI adapters, and the like to the degree that it will support them.

quentin.thebault_defenso.fr added a subscriber: quentin.thebault_defenso.fr.Thu, Sep 3, 12:29 AM

Requesting changes from earlier manpage review

usr.sbin/bhyve/bhyve.8
1340 ↗	(On Diff #181820)	In addition to the cursor sync that was already on that line?

This revision now requires changes to proceed.Fri, Sep 11, 1:55 PM

sorry if i am testing the diff the wrong way and this is not the right place to report test results,
i am having some difficulties getting it working with usb 2.0 hi speed mass storage devices,i have tried three different USBs
first when i was trying to compile the diff,i got hit with

/usr/src/usr.sbin/bhyve/usb_passthru.c:493:15: error: use of undeclared identifier 'LIBUSB_OPTION_CAPSICUMIZE'
  493 |                 { .option = LIBUSB_OPTION_CAPSICUMIZE }
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~

then kevans@ informed me that option has been scrapped in favour of libusb doing sandboxing internally
then when i passthrough a usb 2.0 hi speed drive

i was getting umass0: <SanDisk Cruzer Blade, class 0/0, rev 2.00/1.00, addr 2> on usbus8
umass0: at uhub1, port 5, addr 2 (disconnected)
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(probe0:umass-sim0:0:0:0): Retrying command, 3 more tries remain
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(probe0:umass-sim0:0:0:0): Retrying command, 2 more tries remain
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(probe0:umass-sim0:0:0:0): Retrying command, 1 more tries remain
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(probe0:umass-sim0:0:0:0): Retrying command, 0 more tries remain
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(probe0:umass-sim0:0:0:0): Error 5, Retries exhausted
umass0: detached
and several other issues such as enumeration and attachment failure during guest startup
performance degradations during bulk transfers
device getting stuck in a reset loop
long(30s) freezes and readahead recoveries during bulk transfer

the problems are in pci_xhci.c and usb_passthru.c mainly

Writes to an unpowered port were dropped entirely, so after a guest power cycle the change bits could never be cleared and the port wedged with the guest hub driver retrying forever. pci_xhci should honor write-1-to-clear of the change bits always, when PP is set, re-initialize the port and post a port status change event so the guest re-enumerates
something like

@@ -358,6 +358,7 @@ static void pci_xhci_dump_trb(struct xhci_trb *trb);
 static void pci_xhci_assert_interrupt(struct pci_xhci_softc *sc);
 static void pci_xhci_reset_slot(struct pci_xhci_softc *sc, int slot);
 static void pci_xhci_reset_port(struct pci_xhci_softc *sc, int portn, int warm);
+static void pci_xhci_init_port(struct pci_xhci_softc *sc, int portn);
 static void pci_xhci_update_ep_ring(struct pci_xhci_softc *sc,
     struct pci_xhci_dev_emu *dev, struct pci_xhci_dev_ep *devep,
     struct xhci_endp_ctx *ep_ctx, uint32_t streamid,
@@ -499,8 +500,24 @@ pci_xhci_portregs_write(struct pci_xhci_softc *sc, uint64_t offset,
 		}
 
 		if ((p->portsc & XHCI_PS_PP) == 0) {
-			WPRINTF(("pci_xhci: portregs_write to unpowered "
-			         "port %d", port));
+			p->portsc &= ~(value &
+			    (XHCI_PS_CSC |
+			     XHCI_PS_PEC |
+			     XHCI_PS_WRC |
+			     XHCI_PS_OCC |
+			     XHCI_PS_PRC |
+			     XHCI_PS_PLC |
+			     XHCI_PS_CEC |
+			     XHCI_PS_CAS));
+
+			if (value & XHCI_PS_PP) {
+				pci_xhci_init_port(sc, port);
+				p->portsc |= XHCI_PS_CSC;
+				pci_xhci_set_evtrb(&evtrb, port,
+				    XHCI_TRB_ERROR_SUCCESS,
+				    XHCI_TRB_EVENT_PORT_STS_CHANGE);
+				pci_xhci_insert_event(sc, &evtrb, 1);
+			}
 			break;
 		}

for example

and another thing is the data path misreported completions three ways. the stall branch set USB_STALL and then overwrote it with processed=1, so a stalled transfer reported as success and the guest never cleared the endpoint halt
Bulk/interrupt completion walked the live ring without stopping at the transfer's USB_LAST_DATA, completing a queued next TD without it reaching the device, the OUT walk stopped early at LINK TRBs or when act_len hit 0, splitting a TD. The errcode byte mixed usb_error_t and enum USB_ERRCODE, the OUT path stored 0/18 and pci_xhci decoded with USB_TO_XHCI_ERR(), so stalls surfaced as wrong codes and failed OUT transfers as success
it should set the stall errcode after the marking loop, bound every completion walk to the submitted TD (captured head/ndata/size, stop at USB_LAST_DATA), store enum USB_ERRCODE values, and decode them explicitly in pci_xhci_xfer_complete. this fixes the mass-storage enumeration reset loop and the throughput degradation.

another issue is that the host kernel re-attaches kernel drivers (umass) to a passed-through device at several points, first the USB explore thread re-probes asynchronously after set-configuration, ugen_set_interface re-attaches synchronously, and libusb_reset_device triggers a full asynchronous re-enumeration. A host driver and the guest then drive the device concurrently and corrupt each other's transfers
i think the right thing would be to remove the libusb_reset_device call from the guest port-reset path since the device keeps its address and configuration, so no host action is needed and add usb_passthru_detach_host_drivers(), called after configuration/interface changes and on port resets, polling until the device stays clean for two consecutive sweeps (the asynchronous re-probe can land between sweeps)

also in usb_passthru.c error codes are stored only via USB_DATA_SET_ERRCODE(data, ...) under if (data) so requests with no data stage (wLength == 0, e.g. a remote-wakeup denial stall, or a control transfer failing before a data block exists) drops the error and the guest sees success. it should report on errblk = data ?: first block instead

additionally libusb_set_configuration cancels all in-flight transfers even when the kernel no-ops the request because the configuration is unchanged
Guests re-issue SET_CONFIGURATION during enumeration and error recovery, killing bulk traffic each time so usb_passthru.c should compare against the active configuration and skip if unchanged

and the last issue i found is that some BOT devices wedge a bulk endpoint (NAK forever, until reset) when a CSW poll arrives long after the data phase and in some scenarios the guest's poll can be delayed past that
The wedged transfer pends until the guest's 30 s SCSI timeout forces a full device reset, causing periodic freezes and throughput collapse from the guest's readahead recovery
so bulk transfers on mass-storage interfaces should be given a 5s timeout and report TIMED_OUT as USB_ERR so the guest runs its fast clear-halt recovery instead(scoped to mass storage only)

please check out the inline comments as well,thank you!

usr.sbin/bhyve/usb_passthru.c
99	it needs a `default` case which returns (USB_ERR_INVAL) so that an unlisted error doesn't fall off and garbage doesn't propagate into xhci event TRBs
653	The scan loop advances cur from xfer->head but tests USB_DATA_OK(xfer, i) with the loop counter, with head != 0 the data-stage block is misidentified or lost.

For posterity, I think we agreed that I would rebase this forward since @aokblast is quite busy these days, and I'm going to see if there are functional pieces I can split this further into. (after Euro)

Taking this to address some feedback

Rebase, small revisions to the manpage, and a variety of cosmetic fixes

Not all feedback has been addressed yet.

Harbormaster completed remote builds in B77012: Diff 186941.Thu, Sep 17, 4:10 AM

ziaee added inline comments.Thu, Sep 17, 2:34 PM

usr.sbin/bhyve/bhyve.8
1340 ↗	(On Diff #181820)	I would actually recommend switching them around so they come in the same order in the explanation as they do in the example

Manpage revision, incorporate some of polyduekes' feedback

Notably, their two inline comments and the note about transitioning a port from
unpowered to powered. I need a little bit to digest the rest, but in the
meantime there are some functional fixes that I'll begin splitting out that are
worth committing separately, even if they aren't problematic until passthru
opens up our diversity beyond the tablet.

Harbormaster completed remote builds in B77035: Diff 186990.Thu, Sep 17, 8:14 PM

The first set of patches breaking this up starts at D59779

bhyve: implement single USB device passthrough support
Needs ReviewPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 179604

usr.sbin/bhyve/Makefile

usr.sbin/bhyve/Makefile.depend

usr.sbin/bhyve/pci_xhci.c

usr.sbin/bhyve/usb_emul.h

usr.sbin/bhyve/usb_emul.c

usr.sbin/bhyve/usb_mouse.c

usr.sbin/bhyve/usb_passthru.c

bhyve: implement single USB device passthrough supportNeeds ReviewPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 179604

usr.sbin/bhyve/Makefile

usr.sbin/bhyve/Makefile.depend

usr.sbin/bhyve/pci_xhci.c

usr.sbin/bhyve/usb_emul.h

usr.sbin/bhyve/usb_emul.c

usr.sbin/bhyve/usb_mouse.c

usr.sbin/bhyve/usb_passthru.c

bhyve: implement single USB device passthrough support
Needs ReviewPublic
Actions

Revision Contents
Changeset List