Page MenuHomeFreeBSD
Differential D52166

bhyve: implement single USB device passthrough support
Needs ReviewPublic
Actions

Authored by aokblast on Aug 26 2025, 3:59 PM.
Tags
None
Referenced Files
F161909564: D52166.id.diff
Tue, Jul 7, 10:45 PM
F161868551: D52166.id160997.diff
Tue, Jul 7, 2:09 PM
F161846046: D52166.diff
Tue, Jul 7, 8:39 AM
F161846024: D52166.diff
Tue, Jul 7, 8:39 AM
Unknown Object (File)
Mon, Jul 6, 6:23 PM
Unknown Object (File)
Sat, Jul 4, 10:50 PM
Unknown Object (File)
Sat, Jul 4, 8:31 PM
Unknown Object (File)
Sat, Jul 4, 6:50 PM
View All Files
Subscribers
2khramtsov_gmail.com
adrian
afedorov
arrowd
avg
bcran
bnovkov
View All 22 Subscribers

Details

Reviewers
None
Group Reviewers
bhyve
manpages
Summary

This patch introduces userspace-only USB passthrough using libusb. It
supports configuring passthrough via a (passthru.vid.pid) tuple.
Currently, only vendor and product ID-based selection is implemented;
selecting specific ugen devices (to differentiate between identical
devices) would require changes to libusb.

All USB transfer types—control, bulk, interrupt, and isochronous—are
supported. Hotplugging is also handled, allowing devices to be unplugged
when guest startup and replugged afterward without issue.

Sponsored by: The FreeBSD Foundation

Test Plan

Everyone can test by -s 30,xhci,passthru.vid.pid,passthru.vid.pid where vid and pid are hex value of vendor id and product id. Only one same device can be passthrough at the same time. Can pass through at most 8 devices.
Or using https://github.com/freebsd/vm-bhyve/pull/36.

Have tested on FreeBSD14, FreeBSD15, Linux Kernel 5.14.8, Windows, UEFI shell:
A USB 1.0 kbd.
A USB 2.0 kbd.
A USB 2.0 kbd + mouse + touchpad.
A USB 3.0 mass storage.
A USB 2.0 bluetooth dongle

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 66818
Build 63701: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
lwhsu added inline comments.Aug 31 2025, 12:55 PM
usr.sbin/bhyve/pci_xhci.c
1174

Is this intentional?

1556–1557

pcif?

aokblast updated this revision to Diff 161294.Aug 31 2025, 5:29 PM

Set PCD for FreeBSD

Harbormaster completed remote builds in B66718: Diff 161294.Aug 31 2025, 5:29 PM
aokblast updated this revision to Diff 161295.Aug 31 2025, 5:30 PM

Set PCD to report port status change

Harbormaster completed remote builds in B66719: Diff 161295.Aug 31 2025, 5:30 PM
aokblast added inline comments.Aug 31 2025, 5:32 PM
usr.sbin/bhyve/pci_xhci.c
1174

I think I will delete this field since Winodows, Linux, FreeBSD does not enable CIE at all.

kenrap_kennethraplee.com added a comment.Aug 31 2025, 10:28 PM
In D52166#1194208, @aokblast wrote:

Hello, could you please try again. I forget to adopt multiple configurations.

With your latest changes, it does prevent bhyve from crashing. Appreciate it!

So, now it seems to create an additional 22 second stall at boot compared to without passing through the ssd with usb adapter.

When fully booted, it seems like it doesn't recognize it as if it weren't passthru'd.

aokblast added a comment.Sep 3 2025, 7:34 AM
In D52166#1194385, @kenrap_kennethraplee.com wrote:
In D52166#1194208, @aokblast wrote:

Hello, could you please try again. I forget to adopt multiple configurations.

With your latest changes, it does prevent bhyve from crashing. Appreciate it!

So, now it seems to create an additional 22 second stall at boot compared to without passing through the ssd with usb adapter.

When fully booted, it seems like it doesn't recognize it as if it weren't passthru'd.

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

obiwac added a subscriber: obiwac.Sep 3 2025, 10:55 AM

I'm still going through the code, but here's a few preliminary comments :)

Thanks for working on this!

usr.sbin/bhyve/pci_xhci.c
2047–2050

dev->dev_ue should always be set no?

2811
usr.sbin/bhyve/usb_emul.h
54

Could you elaborate on what this is in a comment? (nit: also missing tab)

usr.sbin/bhyve/usb_passthru.c
32

imo is_in or something would be clearer. But also, what value should a control endpoint have here, since it is bidirectional?

224

Should be after done: label

424

This isn't the only failure case for libusb_init_context, right? I fear this error might be too specific and cause confusion. Maybe you wanna use libusb_error_name here.

435–441

perhaps we should do some input validation here for param1/2

446

nit: vid=%0lx pid=%0lx would make things more obvious for people encountering this error message

458–462

You can use UD_IS_USB2/3 here (this just checks the major so you're fine w.r.t. >=).

Also, is it fine to set the USB version to 2 if bcdUSB is 0x0110 e.g.?

463

nit: I kinda feel like an explicit configured member per endpoint would be clearer instead of setting the type values to -1. But I don't feel strongly about this

494–495

You are turning the libusb error code into a USB error code in usb_passthru_guest_attach_device with libusb_error_to_usb_error, so guessing you meant to check != USB_ERR_NORMAL_COMPLETION

806–807

Clearer if type & 3 was UE_GET_XFERTYPE. Didn't find an equivalent macro for checking if type is in/out, but it would be nice to add one if doesn't exist

kenrap_kennethraplee.com added a comment.Sep 3 2025, 12:33 PM
In D52166#1195280, @aokblast wrote:

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

Quick question. How can I go about setting those options and generating the report using vm-bhyve? I already patched it using the diff from your github PR.

aokblast added a comment.Sep 3 2025, 1:30 PM
In D52166#1195391, @kenrap_kennethraplee.com wrote:
In D52166#1195280, @aokblast wrote:

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

Quick question. How can I go about setting those options and generating the report using vm-bhyve? I already patched it using the diff from your github PR.

Hello, sorry it is from the code directly since it works like the normal DPRINTF. xhci_debug is in pci_xhci.c. usb_passthru_debug is in usb_passthru.c

kenrap_kennethraplee.com added a comment.Sep 3 2025, 2:15 PM
In D52166#1195408, @aokblast wrote:
In D52166#1195391, @kenrap_kennethraplee.com wrote:
In D52166#1195280, @aokblast wrote:

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

Quick question. How can I go about setting those options and generating the report using vm-bhyve? I already patched it using the diff from your github PR.

Hello, sorry it is from the code directly since it works like the normal DPRINTF. xhci_debug is in pci_xhci.c. usb_passthru_debug is in usb_passthru.c

Gotcha, thanks for clarifying: Here is the report (3911 lines): https://gist.github.com/kenrap/ecfc003562bd8717900a46de4903d7bf

aokblast updated this revision to Diff 161435.Sep 3 2025, 4:06 PM

Fixes from comment

Harbormaster completed remote builds in B66784: Diff 161435.Sep 3 2025, 4:06 PM
aokblast added a comment.Sep 3 2025, 4:07 PM

Thanks @obiwac!

usr.sbin/bhyve/usb_passthru.c
32

Since it is for data only (bulk, intr, and isoc). Therefore, we don't have to care about control endpoint in here. For control xfer (endpoint 1), we will call ue_request instead of ue_data. I think a control endpoint other than ep number 1 is undefined behavior.

458–462

The xhci itself have two controller. One is for usb 1.0, 2.0 devices, another is for usb 3 devices. Setting the value to here is just for helping us know which controller should we use. If you open in a Windows Guest. You will see there is a xHCI and a eHCI even if we only have a xHCI instance. So setting the value 2 is fine here.
Also, the bcdUSB returns from the libusb is just a uint16_t instead of a uint8_t[2], so I think UD_IS_USB2 will not work?

aokblast added a comment.Sep 3 2025, 4:07 PM
In D52166#1195426, @kenrap_kennethraplee.com wrote:
In D52166#1195408, @aokblast wrote:
In D52166#1195391, @kenrap_kennethraplee.com wrote:
In D52166#1195280, @aokblast wrote:

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

Quick question. How can I go about setting those options and generating the report using vm-bhyve? I already patched it using the diff from your github PR.

Hello, sorry it is from the code directly since it works like the normal DPRINTF. xhci_debug is in pci_xhci.c. usb_passthru_debug is in usb_passthru.c

Gotcha, thanks for clarifying: Here is the report (3911 lines): https://gist.github.com/kenrap/ecfc003562bd8717900a46de4903d7bf

Could you please check if it works now?

kenrap_kennethraplee.com added a comment.EditedSep 3 2025, 5:42 PM
In D52166#1195497, @aokblast wrote:
In D52166#1195426, @kenrap_kennethraplee.com wrote:
In D52166#1195408, @aokblast wrote:
In D52166#1195391, @kenrap_kennethraplee.com wrote:
In D52166#1195280, @aokblast wrote:

Hello, could you please set the xhci_debug=1 and usb_passthru_debug=1 in bhyve and send me the report? The report maybe very long I think.

Quick question. How can I go about setting those options and generating the report using vm-bhyve? I already patched it using the diff from your github PR.

Hello, sorry it is from the code directly since it works like the normal DPRINTF. xhci_debug is in pci_xhci.c. usb_passthru_debug is in usb_passthru.c

Gotcha, thanks for clarifying: Here is the report (3911 lines): https://gist.github.com/kenrap/ecfc003562bd8717900a46de4903d7bf

Could you please check if it works now?

No change in behavior. Here is a new report: https://gist.github.com/kenrap/a39b327616560bfd5edb816960953d4d

aokblast updated this revision to Diff 161514.Sep 4 2025, 2:01 PM

Fix primary stream id failed

Harbormaster completed remote builds in B66817: Diff 161514.Sep 4 2025, 2:01 PM
aokblast added a comment.Sep 4 2025, 2:02 PM

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

kenrap_kennethraplee.com added a comment.Sep 4 2025, 2:06 PM
In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

kenrap_kennethraplee.com added a comment.Sep 4 2025, 2:53 PM

No difference. The report: https://gist.github.com/kenrap/9a1c406d097aa12c7a7b7ebe106cb681

aokblast updated this revision to Diff 161520.Sep 4 2025, 3:34 PM

Release max number of Pstreams

Harbormaster completed remote builds in B66818: Diff 161520.Sep 4 2025, 3:34 PM
aokblast added a comment.Sep 4 2025, 3:35 PM
In D52166#1196057, @kenrap_kennethraplee.com wrote:
In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

kenrap_kennethraplee.com added a comment.Sep 4 2025, 4:30 PM
In D52166#1196089, @aokblast wrote:
In D52166#1196057, @kenrap_kennethraplee.com wrote:
In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

So, there is good news and bad news.

Good news is, the boot time went down significantly to 6 seconds.

The bad news, the device is still not detected by the guest OS.

Another report: https://gist.github.com/kenrap/2ad9fc1a216e0c6f7eb089c4f3b44c5f

aokblast added a comment.EditedSep 4 2025, 5:03 PM
In D52166#1196117, @kenrap_kennethraplee.com wrote:
In D52166#1196089, @aokblast wrote:
In D52166#1196057, @kenrap_kennethraplee.com wrote:
In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

So, there is good news and bad news.

Good news is, the boot time went down significantly to 6 seconds.

The bad news, the device is still not detected by the guest OS.

Another report: https://gist.github.com/kenrap/2ad9fc1a216e0c6f7eb089c4f3b44c5f

In D52166#1196117, @kenrap_kennethraplee.com wrote:
In D52166#1196089, @aokblast wrote:
In D52166#1196057, @kenrap_kennethraplee.com wrote:
In D52166#1196056, @aokblast wrote:

Sorry, I know that it is a little bit annoying. But since I don't have such device. Could you please try again?

No worries, I want to make sure this gets done right. I'll be your beta tester for this. 👍

I'm rebuilding my src tree now.

Thanks. I really need to get a device with stream in their descriptor. Could you please try again.

So, there is good news and bad news.

Good news is, the boot time went down significantly to 6 seconds.

The bad news, the device is still not detected by the guest OS.

Another report: https://gist.github.com/kenrap/2ad9fc1a216e0c6f7eb089c4f3b44c5f

Hello: Does your device work on FreeBSD host? I saw a transfer is cancelled so the problem maybe on the libusb side or kernel stack.

kenrap_kennethraplee.com added a comment.Sep 4 2025, 11:51 PM
In D52166#1196153, @aokblast wrote:

Hello: Does your device work on FreeBSD host? I saw a transfer is cancelled so the problem maybe on the libusb side or kernel stack.

Sorry for the late reply. Yes, I'm able to mount the device on my FreeBSD host.

aokblast added a comment.EditedSep 7 2025, 5:05 PM

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

kenrap_kennethraplee.com added a comment.Sep 7 2025, 5:13 PM
In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

bz added a comment.Sep 11 2025, 2:03 AM

Hi, I've tested this with a licensing dongle and a smart card reader on an external USB HUB with Windows 11. Licensing worked, smart card was detected correctly. I am still wondering if param1/2 in the config are the best thing but they work, so be it. Many thanks to you and the FreeBSD Foundation! I'll keep trying and using it for other bits as well the next days on other machines. I just need to disable the debugging as my remote window kept scrolling (have the tablet on for vnc as well in this case) like I was on a 9600 connection downloading Netscape ;-)

avg added a subscriber: avg.Nov 11 2025, 3:13 PM
olevole_olevole.ru added a subscriber: olevole_olevole.ru.Nov 21 2025, 12:38 PM
bz added a comment.Nov 26 2025, 11:56 PM

It's gotten silent here. May I ask what the plan is? Is there a chance to get this work into 16?

aokblast added a comment.EditedNov 29 2025, 4:57 AM
In D52166#1232521, @bz wrote:

It's gotten silent here. May I ask what the plan is? Is there a chance to get this work into 16?

Hello. I received the notice of conscription for 4 month obligatory military service in my country in late Octorber which have started from November. I will continue to work on this after my service.

aokblast updated this revision to Diff 173134.Mar 4 2026, 3:31 PM

Rebase to main

Harbormaster completed remote builds in B71181: Diff 173134.Mar 4 2026, 3:31 PM
aokblast added a comment.Mar 4 2026, 3:40 PM
In D52166#1196957, @kenrap_kennethraplee.com wrote:
In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

kenrap_kennethraplee.com added a comment.Mar 4 2026, 10:38 PM
In D52166#1273524, @aokblast wrote:
In D52166#1196957, @kenrap_kennethraplee.com wrote:
In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

Welcome back! 👋

And thanks for rebasing too. I upgraded my base for the updated patch and others as well.

Sadly the non-detection problem still persists...

I got question though. Is there a way to passthru with a specific ugen number of the same vendor and product IDs?

I'm using multiple of these adapters and they have the same IDs. Only the earliest ugen device gets passthru'd when I need a later one instead.

I've been out of the loop with this so I'm trying to recall what I've forgotten. 🙂

aokblast added a comment.Mar 5 2026, 5:39 AM
In D52166#1273749, @kenrap_kennethraplee.com wrote:
In D52166#1273524, @aokblast wrote:
In D52166#1196957, @kenrap_kennethraplee.com wrote:
In D52166#1196949, @aokblast wrote:

Hello, I think I need to at least get a usb 3 hub to test this feature since I haven’t had any super speed device. I will take a look at the candidates hub. However, I am occupied by other stuff recently. Maybe you won’t get any feedback very soon (Maybe a months afterI promise that I will be back ASAP since I would like to finish this feature. Sorry for your inconvenience on helping me test this!

Appreciate the heads up. I'll be on standby.

Hello, I am back and I suspect that https://reviews.freebsd.org/D55289 fixes the problem.
I have rebased the patch to the HEAD.
Could you please check if it actually works?

Welcome back! 👋

And thanks for rebasing too. I upgraded my base for the updated patch and others as well.

Sadly the non-detection problem still persists...

I got question though. Is there a way to passthru with a specific ugen number of the same vendor and product IDs?

I'm using multiple of these adapters and they have the same IDs. Only the earliest ugen device gets passthru'd when I need a later one instead.

I've been out of the loop with this so I'm trying to recall what I've forgotten. 🙂

Sure, it is possible by just specify ugen number. I can work on this.
And I need some time to take a look at what I was doing 4 months ago:)

bz added a comment.Mar 5 2026, 8:30 AM

Woohoo. Happy to see updates here.

I've rebased my main once in between; while my smart card reader still works on Win11 after passthru one of my licensing dongles no longer does. I'll grab an updated stack from here the next days and rebuild to see. I haven't had time to debug it anyway so can as well do it on the latest revision. Thanks for all this work!

2khramtsov_gmail.com added a subscriber: 2khramtsov_gmail.com.Mar 6 2026, 2:58 AM
o_fornost.net added a subscriber: o_fornost.net.Mar 17 2026, 5:58 PM
had-churn-float_duck.com added a subscriber: had-churn-float_duck.com.Apr 20 2026, 11:09 AM
aokblast added a comment.May 21 2026, 1:52 PM
In D52166#1273856, @bz wrote:

Woohoo. Happy to see updates here.

I've rebased my main once in between; while my smart card reader still works on Win11 after passthru one of my licensing dongles no longer does. I'll grab an updated stack from here the next days and rebuild to see. I haven't had time to debug it anyway so can as well do it on the latest revision. Thanks for all this work!

Hi, I found a very subtle fails in our xHCI. Please also apply this patch while you are testing.

https://reviews.freebsd.org/D57146

emaste added a subscriber: emaste.Mon, Jun 8, 4:40 PM
aokblast updated this revision to Diff 179604.Thu, Jun 11, 3:00 PM

Large fixes

Harbormaster completed remote builds in B73821: Diff 179604.Thu, Jun 11, 3:00 PM
aokblast added a comment.Thu, Jun 11, 3:10 PM

I did a very large regions of bug fixes.

The full patchset is in here https://github.com/aokblast/freebsd-src/tree/bhyve/impl_usbdev_passthru. It should fix many unworking devices.

aokblast edited the test plan for this revision. (Show Details)Thu, Jun 11, 3:10 PM
adrian added a subscriber: adrian.Thu, Jun 11, 4:17 PM

in general, nice work!

usr.sbin/bhyve/usb_passthru.c
763

is err >= 8 here validating the payload length?

aokblast added inline comments.Thu, Jun 11, 4:23 PM
usr.sbin/bhyve/usb_passthru.c
763

Yes, see document in https://libusb.sourceforge.io/api-1.0/group__libusb__syncio.html. I might need to rename the variable name.

ziaee added a comment.Fri, Jun 12, 2:44 PM

please also add to manpage in this commit

aokblast updated this revision to Diff 179653.Fri, Jun 12, 4:11 PM

Add doc

Harbormaster completed remote builds in B73841: Diff 179653.Fri, Jun 12, 4:12 PM
ziaee added a reviewer: manpages.Fri, Jun 12, 4:49 PM

Thanks! note that patch does not apply to main

usr.sbin/bhyve/bhyve.8
1070 ↗(On Diff #179653)

note that the existing Ar tablet is also wrong, tablet is not a variable argument, its a command modifier.

1075 ↗(On Diff #179653)
adrian added a comment.Fri, Jul 3, 12:01 AM

Hey, any further traction on this? What's left before it can be landed?

zldrobit_gmail.com added a subscriber: zldrobit_gmail.com.Fri, Jul 3, 9:06 AM
bnovkov added a subscriber: bnovkov.Tue, Jul 7, 8:05 PM
bnovkov added inline comments.
usr.sbin/bhyve/pci_xhci.c
2892–2901
usr.sbin/bhyve/usb_passthru.c
416–421

Broken formatting?

503

Should you defer setting this until after you've checked the error?

581

We should check the return value here and for pthread_mutexattr_init and destroy the mutex in failed: only if it was initialized.

636

bhyve/usb_mouse.c defines the same macro, would it make sense to split it out into a common header?

814–815

Saving a pointer to sc->endpoint_types[epctx << 1 | dir] would make more sense given that you repeat a non-trivial calculation three times in this function.

930–931

Revision Contents
Changeset List

PathSize
usr.sbin/
bhyve/
3 lines
1 line
260 lines
22 lines
1 line
25 lines
871 lines

Diff 161520

View Options

usr.sbin/bhyve/Makefile

Loading...
View Options

usr.sbin/bhyve/Makefile.depend

Loading...
View Options

usr.sbin/bhyve/pci_xhci.c

Loading...
View Options

usr.sbin/bhyve/usb_emul.h

Loading...
View Options

usr.sbin/bhyve/usb_emul.c

Loading...
View Options

usr.sbin/bhyve/usb_mouse.c

Loading...
View Options

usr.sbin/bhyve/usb_passthru.c

Loading...