usb: skip root_mount_hold when root is not on USB

[guest-seuros]

Summary

USB controllers unconditionally call root_mount_hold() during attach,
delaying boot by several seconds while USB buses enumerate even when
the root filesystem is on SATA, NVMe, or ZFS (which are never USB).

The existing hw.usb.no_boot_wait tunable still works as an override.

[ngie]

the root filesystem is on SATA, NVMe, or ZFS (which are never USB).

That's a bit of a bold claim because ZFS can use USB devices. Is it smart? Hell no. Is it possible? Yes. Also, you would need to deal with all GEOM layers, e.g., graid, gmirror, etc.

I understand what you're trying to solve, but I'm a bit skeptical of the approach being taken given how much things can be stacked on top of one another in FreeBSD from a storage perspective between cam (ctl anyone?), geom, and ZFS.

[ngie]

Back in the day this also used to be a flat sleep for all of cam.. it was worse than it is currently.
I feel like better dependency tracking needs to be done to avoid this behavior.
Also... this might be deal with/benefit ukbd as back in the good old days (when there was a flat sleep), the mount root prompt would be unusable with USB keyboards, leaving the sysadmin with a wedged computer terminal asking for a root device and not being able to actually input anything on the keyboard to remedy that situation. Wash-rinse-repeat and the reboot cycle increases because power cycling sometimes reinitializes firmware and has to respin up drives in JBODs, etc (especially in the legacy BIOS and old UEFI days).
So yes, this is an inconvenience, but I would temper removing this based only on the names of functions, etc, alone.
Please go through git blame and see whether or not the change that added this code (functionally, not restylized it) was dealing with the case we think it should be dealing with (quiescing USB root devices) instead of dealing with the vfs_mountroot prompt.

[ngie]

In D58869#1351234, @ngie wrote:

Back in the day this also used to be a flat sleep for all of cam.. it was worse than it is currently.
I feel like better dependency tracking needs to be done to avoid this behavior.
Also... this might be deal with/benefit ukbd as back in the good old days (when there was a flat sleep), the mount root prompt would be unusable with USB keyboards, leaving the sysadmin with a wedged computer terminal asking for a root device and not being able to actually input anything on the keyboard to remedy that situation. Wash-rinse-repeat and the reboot cycle increases because power cycling sometimes reinitializes firmware and has to respin up drives in JBODs, etc (especially in the legacy BIOS and old UEFI days).
So yes, this is an inconvenience, but I would temper removing this based only on the names of functions, etc, alone.
Please go through git blame and see whether or not the change that added this code (functionally, not restylized it) was dealing with the case we think it should be dealing with (quiescing USB root devices) instead of dealing with the vfs_mountroot prompt.

Please note that I have a fair bit of frustration/PTSD from dealing with bricked FreeBSD boxes stuck on vfs_mountroot prompts--in particular with USB keyboards or with BIOSes that were "smart" and powered off PS/2 controllers if PS/2 devices were not present at boot. It only got worse once I did a fair bit of testing with 32+ bay JBODs back at a previous job where running into this scenario and not having a soft reset option would trigger the pathological power cycling case I mentioned earlier, largely because powercycling at outlets generally meant flipping the circuit on and off instead of using other means like IPMI to power cycle instead using soft reset, etc.

[guest-seuros]

adress comments

[guest-seuros]

In D58869#1351233, @ngie wrote:

the root filesystem is on SATA, NVMe, or ZFS (which are never USB).

That's a bit of a bold claim because ZFS can use USB devices. Is it smart? Hell no. Is it possible? Yes. Also, you would need to deal with all GEOM layers, e.g., graid, gmirror, etc.

I understand what you're trying to solve, but I'm a bit skeptical of the approach being taken given how much things can be stacked on top of one another in FreeBSD from a storage perspective between cam (ctl anyone?), geom, and ZFS.

I forgot that we could have ZFS in a mounted enclosure. I removed the filesystem detections.

[imp]

I think this is generally a really bad idea.

> kenv | grep vfs
vfs.root.mountfrom="ufs:/dev/ada0p2"
vfs.root.mountfrom.options="rw"

So we almost always have a : in the path, since the boot loader adds it.
And I think it breaks asking for a root device (say if the root is supposed to be nda0, but that's broken, we may not have the thumb drive you plugged in detected before we get to mountroot prompt).
And many people have zpools on usb, even when booting from ada, so their pools will disappear if the boot path is faster.

Also, CAM has always waited for the root hold.

There's two real issues here: (1) Cam can be smarter about the boot hold: it waits for all the SIMs to probe their buses, but isn't. It's easy to speed up the process, but that speedup comes at the cost of stable device naming. (2) the waiting we do in the mountroot code could also be smarter and just wait until root shows up, but then you have the 'other filesystems' issue.
The current tunable to skip waiting for USB is about as far as we can go, I think. If USB is holding things up, then usb needs to be fixed.