Page MenuHomeFreeBSD

Support for MSI-X on Annapurna Alpine
ClosedPublic

Authored by mst_semihalf.com on Aug 19 2016, 11:36 AM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Nov 23, 10:37 AM
Unknown Object (File)
Fri, Nov 22, 5:41 AM
Unknown Object (File)
Thu, Nov 21, 11:55 AM
Unknown Object (File)
Tue, Nov 12, 4:20 AM
Unknown Object (File)
Tue, Nov 12, 4:11 AM
Unknown Object (File)
Mon, Nov 11, 11:26 AM
Unknown Object (File)
Tue, Nov 5, 4:03 AM
Unknown Object (File)
Tue, Nov 5, 4:01 AM

Details

Summary

This patch adds support for MSI-X interrupts on Annapurna Alpine platform. MSI-X on Alpine work similarly to GICv2m, i.e. some range of SPI interrupts is reserved in GIC and individual SPIs can be triggered by MSI-X messages. This SPI range is defined in FDT.

I left in support for non-INTRNG kernel as INTRNG was not enabled on ARM64 at the time of writing this driver. However, I can remove it if INTRNG is to stay.
EDIT: I removed non-INTRNG parts of the driver.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

mst_semihalf.com retitled this revision from to Support for MSI-X on Annapurna Alpine.
mst_semihalf.com updated this object.
mst_semihalf.com edited the test plan for this revision. (Show Details)
mst_semihalf.com set the repository for this revision to rS FreeBSD src repository - subversion.
mst_semihalf.com added subscribers: ARM, arm64.

However, I can remove it if INTRNG is to stay.

There is no way to disable INTRNG and get a working arm64 kernel.

However, I can remove it if INTRNG is to stay.

There is no way to disable INTRNG and get a working arm64 kernel.

This driver is also used on ARM32 (there's a 32-bit Alpine SoC with the same devices) but INTRNG is enabled there as well so I'll remove non-INTRNG parts to make the patch easier to review.

mst_semihalf.com updated this object.

Remove non-INTRNG parts of the driver as they are no longer necessary.

Use device entry instead of SOC option.

Is there any reason these bindings diverge from the ones from the vendor? http://lxr.free-electrons.com/source/arch/arm/boot/dts/alpine.dtsi

mst_semihalf.com edited edge metadata.

Rework MSI-X allocation to suit post-D7493 INTRNG.
I also removed some oddities leftover from old versions of the driver and simplified the code.

I added a small change to GICv3 (D7662) as BUS_SETUP_INTR was failing in the case of MSIX interrupts which were not extended with FDT description.

Move MSIX file entry from sys/conf/files to files.arm and files.arm64. Add fdt dependency.

sys/arm/annapurna/alpine/alpine_pci_msix.c
161

Shouldn't this be an error? When we fail with registering device, we end up with non-working MSI/MSIx for the whole system and the driver successfully loaded.

Return error when MSI could not be registered.

sys/arm/annapurna/alpine/alpine_pci_msix.c
161

Fixed

sys/arm/annapurna/alpine/alpine_pci_msix.c
207–209

You seem to be requesting a contiguous range of interrupts and mapping them 1:1. This is the sort of thing intr_pic_add_handler was designed to handle. You might need to tell the GICv3 driver to send this range to the child.

314–321

Why not use vmem(9) to manage this? I've done so in the GICv3 ITS driver if you would like a reference.

326–333

Can you not use ofw_bus_map_intr?

sys/arm/annapurna/alpine/alpine_pci_msix.c
207–209

I know, but currently the child handler is called only in the case of LPIs so it only works with ITS on ARM64. I did not want to introduce delay in arm_gic_v3_intr() by adding additional branches, but perhaps it should somehow be resolved so that not only ITS is supported as child. Another potential issue is that the 'msix' node here is not really a child of GICv3 node in DTS - it is at the same level under 'soc'.

326–333

That only decodes the IRQ number, while PIC_MAP_INTR provides the isrc from GIC which we can return to the caller.

Change MSI-X number allocation scheme to make use of vmem_alloc() instead of local bitmap.

sys/arm/annapurna/alpine/alpine_pci_msix.c
314–321

Diff 21508 switches management to vmem. Had some trouble with it because vmem_alloc() seems to eventually fail when smallest unit of allocation is set to 1 and qcache_max is also 1. In ITS you always allocated more than one element at a time and I suppose that's why it worked for you - I think you should look into what happens when you try to allocate only one item. In my case a few initial allocations were successful but then it returned ENOMEM even though there was still a lot of free elements.

wma edited edge metadata.

Seems fixed

This revision is now accepted and ready to land.Oct 20 2016, 11:23 AM
This revision was automatically updated to reflect the committed changes.