Does this mean all TB3 support will be removed? Is there more to cleanup in that case?

Does this mean all TB3 support will be removed?

as I understood it, USB4 pretty much just standardized existing TB3 controllers, so I guess it would be possible to have a TB3 controller for which a USB4 driver/HCM would work. Not that I know of any such controllers. This is the existing check to set HCM usage, which would maybe imply there are some devices which normally use an ICM but for which a HCM would work:

	if (NHI_IS_USB4(sc) || (sc->force_hcm == NHI_FORCE_HCM_ON))
		sc->caps |= NHI_CAP_HCM;

maybe @scottl could elucidate this

Is there more to cleanup in that case?

yeah, we should anyway also remove all this force_hcm stuff seeing as that's all we're going to support.

The relationship between TB3 and USB4 isn't as simple as you suggest. There are optional vs mandatory feature considerations, there are differences in the bus properties and device config registers, there are differences in the necessary behavior of the Connection Manager, not to mention that an HCM is mandatory for USB4 but not for TB3, and there are even differences in the NHI programming interface. A better comparison is TB4 and USB4, their programming interfaces and functional attributes are more closely related, with TB4 being a superset of USB4.

If you've removed the ICM code, then don't even think about supporting TB3 hosts in the future. However, the tb_pci and picb devices are kind of special. They exist in device peripherals, not just hosts, and there are a lot of devices out there that use AR as their internal controller. Supporting them doesn't mean that you have to support an AR NHI controller. If you want to remove the code from freebsd, that's fine I guess, but it was nice to have the tb pci devices clearly identified in the device tree and dmesg. Also, keep in mind that if you support TB3 devices, your USB4 NHI and HCM code will need to explicitly support them.

Honestly, I'm kinda confused about the state of this code in freebsd. The ICM code was removed, which removes any control that the OS has over TB3 controllers and topology. An HCM doesn't exist yet, correct(?), and neither does USB4-specific enablement code in NHI? If so, then what functionality does this code have in the freebsd repo at the moment? Don't get me wrong, I'm glad that people are interested in this code and in USB4, but there was also a reason that I didn't commit this code back in 2022, it wasn't anywhere close to being ready yet =-)

In D52861#1273135, @scottl wrote:

...

Honestly, I'm kinda confused about the state of this code in freebsd. The ICM code was removed, which removes any control that the OS has over TB3 controllers and topology. An HCM doesn't exist yet, correct(?), and neither does USB4-specific enablement code in NHI? If so, then what functionality does this code have in the freebsd repo at the moment? Don't get me wrong, I'm glad that people are interested in this code and in USB4, but there was also a reason that I didn't commit this code back in 2022, it wasn't anywhere close to being ready yet =-)

Wait... the TB3/ICM code was removed from FreeBSD...? If so, that completely broke any hope of being able to use TB on my Intel MacBook hardware :(. My MacBookPro13,1, MacBookPro16,1, and Macmini8,1 all rely on TB3 to function :(.

The relationship between TB3 and USB4 isn't as simple as you suggest. There are optional vs mandatory feature considerations, there are differences in the bus properties and device config registers, there are differences in the necessary behavior of the Connection Manager, not to mention that an HCM is mandatory for USB4 but not for TB3, and there are even differences in the NHI programming interface. A better comparison is TB4 and USB4, their programming interfaces and functional attributes are more closely related, with TB4 being a superset of USB4.

Got it, thanks for clearing that up. My previous impression was that USB4 was a superset of TB3. Do you have a specific source for this or some comprehensive overview? It's been a little while since I've worked on this but all I really went off was the USB4 spec and didn't find too much info on TB3, and the only hardware I was really concerned about was USB4-only anyways.

If you've removed the ICM code, then don't even think about supporting TB3 hosts in the future. However, the tb_pci and picb devices are kind of special. They exist in device peripherals, not just hosts, and there are a lot of devices out there that use AR as their internal controller. Supporting them doesn't mean that you have to support an AR NHI controller. If you want to remove the code from freebsd, that's fine I guess, but it was nice to have the tb pci devices clearly identified in the device tree and dmesg. Also, keep in mind that if you support TB3 devices, your USB4 NHI and HCM code will need to explicitly support them.

The ICM code can be added back later down the road if we want to support this.

Honestly, I'm kinda confused about the state of this code in freebsd. The ICM code was removed, which removes any control that the OS has over TB3 controllers and topology. An HCM doesn't exist yet, correct(?), and neither does USB4-specific enablement code in NHI? If so, then what functionality does this code have in the freebsd repo at the moment? Don't get me wrong, I'm glad that people are interested in this code and in USB4, but there was also a reason that I didn't commit this code back in 2022, it wasn't anywhere close to being ready yet =-)

My main purpose for bringing this code in was to be able to suspend the USB4 controller, which is necessary for S0i3 entry on certain platforms (due to a "pre-OS" connection manager leaving the controller on when the OS boots by default on these platforms - see D49453). At some point I would like to work on getting a full HCM written though.

Wait... the ICM code was removed from FreeBSD...? If so, that completely broke any hope of being able to use TB on my Intel MacBook hardware :(. My MacBookPro13,1, MacBookPro16,1, and Macmini8,1 all rely on TB3 to function :(.

It was never in FreeBSD; I left it out in my original import of the USB4 (D49450). The bits this revision remove are bits that I probably should've removed during the initial import but were left over. But if someone is willing to work on support for TB3, then it can totally be brought in!

In D52861#1273156, @obiwac wrote:

...

If you've removed the ICM code, then don't even think about supporting TB3 hosts in the future. However, the tb_pci and picb devices are kind of special. They exist in device peripherals, not just hosts, and there are a lot of devices out there that use AR as their internal controller. Supporting them doesn't mean that you have to support an AR NHI controller. If you want to remove the code from freebsd, that's fine I guess, but it was nice to have the tb pci devices clearly identified in the device tree and dmesg. Also, keep in mind that if you support TB3 devices, your USB4 NHI and HCM code will need to explicitly support them.

The ICM code can be added back later down the road if we want to support this.

Honestly, I'm kinda confused about the state of this code in freebsd. The ICM code was removed, which removes any control that the OS has over TB3 controllers and topology. An HCM doesn't exist yet, correct(?), and neither does USB4-specific enablement code in NHI? If so, then what functionality does this code have in the freebsd repo at the moment? Don't get me wrong, I'm glad that people are interested in this code and in USB4, but there was also a reason that I didn't commit this code back in 2022, it wasn't anywhere close to being ready yet =-)

My main purpose for bringing this code in was to be able to suspend the USB4 controller, which is necessary for S0i3 entry on certain platforms (due to a "pre-OS" connection manager leaving the controller on when the OS boots by default on these platforms - see D49453). At some point I would like to work on getting a full HCM written though.

Wait... the ICM code was removed from FreeBSD...? If so, that completely broke any hope of being able to use TB on my Intel MacBook hardware :(. My MacBookPro13,1, MacBookPro16,1, and Macmini8,1 all rely on TB3 to function :(.

It was never in FreeBSD; I left it out in my original import of the USB4 (D49450). The bits this revision remove are bits that I probably should've removed during the initial import but were left over. But if someone is willing to work on support for TB3, then it can totally be brought in!

I see.

If I had access to the specs, I would like to get ICM/HCM working (which includes investing time in making that functionality in TB3 work). It's mildly annoying how I need to have separate RJ-45 dongles for communication between TB nodes when (in reality) I could just get 2 hosts to talk directly with each other over another IP-like protocol using a TB3/4 capable cable.

@guest-seuros and I are working on getting Intel Mac hardware working on FreeBSD. Getting something like ICM in would be helpful and help with interop between FreeBSD and macOS. Once later gen Intel Mac hardware works (iBridge, SMC support, etc), moving to Apple Silicon based platforms should be easier to do.

If I had access to the specs, I would like to get ICM/HCM working (which includes investing time in making that functionality in TB3 work). It's mildly annoying how I need to have separate RJ-45 dongles for communication between TB nodes when (in reality) I could just get 2 hosts to talk directly with each other over another IP-like protocol using a TB3/4 capable cable.

I guess we should first make sure the hardware you have doesn't work with a HCM. I'm assuming Linux has an equivalent of the hw.nhi.force_hcm tunable this revision removes that you could use to test. If not, then I can add the ICM code in (and work on cleaning up/landing my subsequent patches :)). Speaking of which, @scottl can I add you as a reviewer on those?

You can get the USB4 spec from here: https://www.usb.org/document-library/usb4r-specification-v20

The two most relevant files are:

• USB4 Connection Manager Guide Ver. 2.0 - rev. 1.1.pdf
• USB4 V2 with Errata and ECN through September 2024- CLEAN.pdf

Things were relatively clear and easy to read as far as specs go.

I must decline being a reviewer or otherwise being involved in the freebsd code. Happy to discuss publicly available information regarding the tech in general.

I think that if you dig deep into the linux nhi and thunderbolt code you'll see some TB3 vs USB4 differences. I know what I know based on my time at Intel and my access to the TB3 specs at the time, obviously I can't share those with you.

In D52861#1273246, @scottl wrote:

I must decline being a reviewer or otherwise being involved in the freebsd code. Happy to discuss publicly available information regarding the tech in general.

I think that if you dig deep into the linux nhi and thunderbolt code you'll see some TB3 vs USB4 differences. I know what I know based on my time at Intel and my access to the TB3 specs at the time, obviously I can't share those with you.

got it :) Thanks for your input so far!

Either return the PCI IDs in the tb_pcib driver, or remove the entire driver. With this review, the entire driver is dead code.

In D52861#1273277, @scottl wrote:

Either return the PCI IDs in the tb_pcib driver, or remove the entire driver. With this review, the entire driver is dead code.

Well no, it still has all the ring and interrupt setup that allows me to read/write to the router config space, which I need for resetting the controller and suspending it (again, my primary reason for bringing this code in now).

And I'll add the Pink Sardine PCI bridge IDs in D52862, which is the only hardware I have to test this on.

Maybe I’m confused, I was referring to tb_pcib.c.

In D52861#1273704, @scottl wrote:

Maybe I’m confused, I was referring to tb_pcib.c.

ah right, yes indeed. Do any of these work without the equivalent NHI support though? I don't have the hardware to test these but my impression was that a connection manager of some sort was necessary to actually configure the bridge to work.