Place an advisory lock on devices opened by libusb_open(3).

[unitrunker_unitrunker.net]

Summary

To more closely match behavior of libusb_open on other platforms, place an advisory lock on the device file descriptor.

Currently, multiple processes can open the same USB device. On Linux and Windows, USB device enumeration omits devices currently in use. A number of ports assume this behavior.

This proposed change will cause libusb_open to report an error (LIBUSB_ERROR_ACCESS) if it finds an advisory lock on the device.

This includes a further change to libusb_open_with_vid_pid to skip over failed attempts to the first successful call to libusb_open.

In an exteme case, this unexpected behavior in the FreeBSD implementation causes two ports to display the same device 32 times (see ports airspy and hydrasdr).

As an example, a FreeBSD host has two USB devices attached that share the same VID/PID but different serial numbers. The intent is ...

a. a first call to libusb_open_with_vid_pid returns a device handle to the first matching device - which the app keeps open.
b. a second call to libusb_open_with_vid_pid returns a device handle to the second matching device (and not the first) - which the app keeps open.
c. a third call to libusb_open_with_vid_pid returns null (and none of the other devices).

libusb_close or process exit remove the advisory lock on the device. The specific advisory lock mechanism is flock(2). A complete patch is included. I will follow up with a test plan and the bits of code used for testing.

Please see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=291506 which includes source and CMakeLists.txt for the usbgrab utility.

Test Plan

A. Single device

1. Attach one device with known VID/PID.
2. run "usbgrab <vid> <pid> -p &" utility to take ownership of the device.
3. Verify the tool displays "Got 1 Press enter ..."
4. run "usbgrab <vid> <pid>" in the foreground.
5. Verify an error is displayed.
6. run "fg" to place the earlier usbgrab process into the foreground.
7. press enter to exit the process.

Verify no usbgrab background processes remain.

B. Two devices, sequential

1. Attach two devices with same known VID/PID.
2. run "usbgrab <vid> <pid> -p &" utility to take ownership of the device.
3. Verify the tool displays "Got 1 Press enter ..."
4. run "usbgrab <vid> <pid>" in the foreground.
5. Verify NO error is displayed.
6. run "fg" to place the earlier usbgrab process into the foreground.
7. press enter to exit the process.

Verify no usbgrab background processes remain.

C. Two devices, all or none

1. Attach two devices with same known VID/PID.
2. run "usbgrab <vid> <pid> -p -a &" utility to take ownership of all matching devices.
3. Verify the tool displays "Got 2 Press enter ..."
4. run "usbgrab <vid> <pid>" in the foreground.
5. Verify an error is displayed.
6. run "fg" to place the earlier usbgrab process into the foreground.
7. press enter to exit the process.

Verify no usbgrab background processes remain.

D. Two devices, alternating

1. Attach two devices with same known VID/PID.
2. run "usbgrab <vid> <pid> -p &" utility to take ownership of the device and pause in the background.
3. Verify the tool displays "Got 1 Press enter ..."
4. Make note of the background process pid.
5. run "usbgrab <vid> <pid> -p &" in the background to take ownership of the second device.
6. Verify "Got 1 Press enter ..." is displayed.
7. run "usbgrab <vid> <pid>" in the foreground.
8. Verify an error is displayed.
9. run "fg <pid>" to place the earlier usbgrab process into the foreground.
10. press enter to exit the process.
11. run "usbgrab <vid> <pid>"
12. Verify "Got 1" is displayed
13. run "fg" to place earlier usbgrab process into foreground.
14. press enter to exit the process.

Verify no usbgrab background processes remain.

[aokblast]

Haven't test it. But I have a comment. It would be great if we can solve multiple device issue like this.

Also, I have another problem. As you have mentioned:

On Linux and Windows, USB device enumeration omits devices currently in use. A number of ports assume this behavior.

Does this happen in libusb layer or udev(winusb) layer? I didn't see any generic handling code in libusb to skip over opened device.

lib/libusb/libusb10.c

651

IMHO, it would be better to move to the libusb backend in libusb20_dev_open as it is a system dependent behavior (even we only have one system).

[unitrunker_unitrunker.net]

In D54173#1238395, @aokblast wrote:

Haven't test it. But I have a comment. It would be great if we can solve multiple device issue like this.

Also, I have another problem. As you have mentioned:

On Linux and Windows, USB device enumeration omits devices currently in use. A number of ports assume this behavior.

Does this happen in libusb layer or udev(winusb) layer?

This is a little complicated. There are two problems that can be solved in different ways.

1. Making access to a USB device exclusive.
2. Enumeration.

For enumeration, I want to know what devices are attached - even if they are in use by another process.
My strong opinion is that exclusivity should be under control of the applications. The host OS should not impose such a restriction.

My understanding is that WinUSB and libusb on Linux both assume exclusive access. For enumeration, Linux omits devices already in use. You can't tell if something is there. WinUSB does the same but there is a cheat. You can scan the registry for currently attached devices so you know something is there even if it is held by another application. On Linux (at least through libusb), you're stuck. In contrast, libusb20 handles these two very well by giving the application control. As an example, I can query the serial number of a device even as it is in use by another process. This is a feature, not a bug.

Pet peeve: dozens of programs enumerate devices in a way where they get a list of opened devices and let the user pick one - and fail to then close access to all the other devices. Users wonder why a connected device is missing in a different program. A terrible experience.

In short, I'd like FreeBSD's libusb to be compatible without compromising libusb20.

I didn't see any generic handling code in libusb to skip over opened device.

I believe this is because the list does not contain opened devices. You might be able to force a race condition by setting a breakpoint and then running a second program that also tries to open the same device. More precisely, if libusb_get_device_list does not give you a libusb_device pointer to a device, you can't call libusb_open to open it. A small test program will verify this behavior.

IMHO, it would be better to move to the libusb backend in libusb20_dev_open as it is a system dependent behavior (even we only have one system).

As stated, this would break a key feature of libusb20. Exclusivity should be optional. I do think something like the O_EXCL flag in POSIX open(2) would be ideal. This would encapsulate the details of the lock mechanism inside libusb20. The problem with libusb20_open is such a parameter doesn't exist.

As long as exclusivity is under client application control, I'm happy.

[unitrunker_unitrunker.net]

Thanks to @aokblast for these comments. I'm thinking of an alternate proposal that moves the exclusivity down to libusb20_dev_open. Something like libusb20_dev_open_exclusive or libusb20_dev_open_with_flags. Opening a device exclusively should appear atomic to the application so it needs to happen inside the open.

[unitrunker_unitrunker.net]

Now I have a dilemma.

I can add exclusivity inside libusb20.c or push down one more layer and add this to the backend in libusb20_ugen20.c. To do this in the backend, device_open_t gets a third parameter for the flag. It's a six-for-one, half-dozen-for-the-other kind of decision.

The scope for this change is exclusivity. No plans to touch existing enumeration behavior.

I'll have to create a separate review. Everything will link back to the same bug.

[aokblast]

In D54173#1238612, @unitrunker_unitrunker.net wrote:

Now I have a dilemma.

I can add exclusivity inside libusb20.c or push down one more layer and add this to the backend in libusb20_ugen20.c. To do this in the backend, device_open_t gets a third parameter for the flag. It's a six-for-one, half-dozen-for-the-other kind of decision.

The scope for this change is exclusivity. No plans to touch existing enumeration behavior.

I'll have to create a separate review. Everything will link back to the same bug.

Hello:

thanks for your detailed reply! I think you are right that flock should be optional in the libusb20 backend. But I am about to back to the military camp today so I am unable to test it right now. I will add USB group to the reviewers so that other people can take a look to it.

[unitrunker_unitrunker.net]

Abandoning in favor of D54218

[unitrunker_unitrunker.net]

lib/libusb/libusb10.c

651

This is done in D54218.