Implement pass-through resource management for ChipCommon

[landon_landonf.org]

Summary

This patchset adds support to bhnd_chipc for sharing SYS_RES_MEMORY resources with
its children, allowing us to hang devices off of bhnd_chipc that rely on access to a subset
of the device register space that bhnd_chipc itself must also allocate.

We could avoid most of this heavy lifting if RF_SHAREABLE+SYS_RES_MEMORY wasn't
limited to use with allocations at the same size/offset.

As a work-around, I implemented something similar to vga_pci.c, which implements similar
reference counting of of PCI BAR resources for its children.

With these changes, chipc will use reference counting of SYS_RES_MEMORY
allocation/activation requests, to decide when to allocate/activate/deactivate/release resources
from the parent bhnd(4) bus.

The requesting child device is allocated a new resource from chipc's rman, pointing to (possibly a
subregion of) the refcounted bhnd resources allocated by chipc.

Other resource types are just passed directly to the parent bhnd bus; RF_SHAREABLE works just
fine with IRQs.

I also lifted the SPROM device code out into a common driver, since this now allows me to
hang simple subclasses off of a common driver off of both bhndb_pci and bhnd_chipc.

Test Plan

Tested against BCM4331 and BCM4312, confirmed that SPROM still
attaches and can be queried.

[mizhka]

sys/dev/bhnd/cores/chipc/chipc.c

773–786

I've tried CHIPC_SPROM_OTP register on board, but it's not mapped into memory.
[BHND debug] chipc_print_capabilities:55 => UARTs: 0x1
[BHND debug] chipc_print_capabilities:56 => BigEngian: 0x0
[BHND debug] chipc_print_capabilities:57 => UART-GPIO: 0x0
[BHND debug] chipc_print_capabilities:58 => UART Clock: 0x1
[BHND debug] chipc_print_capabilities:59 => Flash type: 0x1
[BHND debug] chipc_print_capabilities:61 => External buses: 0x0
[BHND debug] chipc_print_capabilities:62 => Power control: 0x0
[BHND debug] chipc_print_capabilities:63 => JTAG master: 0x1
[BHND debug] chipc_print_capabilities:65 => PLL Type: 0x0
[BHND debug] chipc_print_capabilities:66 => OTP size: 0x1
[BHND debug] chipc_print_capabilities:67 => Is 64bit? 0x0
[BHND debug] chipc_print_capabilities:68 => Boot ROM: 0x0
[BHND debug] chipc_print_capabilities:69 => PMU: 0x1
[BHND debug] chipc_print_capabilities:70 => ECI: 0x0
[BHND debug] chipc_print_capabilities:71 => SPROM: 0x0
[BHND debug] chipc_print_capabilities:72 => NFLASH: 0x0
bhnd_chipc0: NVRAM-OTP unsupported
...
db> x/wx 0xb8000800
0xb8000800:Trap cause = 7 (bus error (load or store) - kernel mode)
panic: trap
panic: mtx_lock() by idle thread 0x8064aa80 on sleep mutex eventhandler @ /repo/freebsd/onion/src/sys/kern/subr_eventhandler.c:251
KDB: enter: panic

So this address can't be used for OTP case.

1208

Typo

1237–1253

Didn't get it. On MIPS router we attach MIPS core driver as earlier as possible, so this code will return false.

1425–1428

D6250 overrides it to RMAN. Is it OK for you?

sys/dev/bhnd/cores/chipc/chipcreg.h

125–128

Too huge. I think it should be less than 0x100, may be 0x30?
http://bcm-v4.sipsolutions.net/ChipCommon

Whatever, OTP registers are between ChipC id and interrupts registers.

[landon_landonf.org]

Inline replies. I'll submit a new diff fixing that typo in a bit.

sys/dev/bhnd/cores/chipc/chipc.c

773–786

Short version, since I accidentally ctrl-W'd my first longer version:

This seems to vary chipset to chipset, and SoC vs WiFi; the ISC-licensed Broadcom code drops are all over the place in terms of when this OTP mapping is used.

However, the addition of a child device here is essentially a no-op if ChipCommon does not advertise BHND_NVRAM_SRC_SPROM, as the bhnd_sprom_chipc driver will not probe and attach.

We can narrow down the OTP/SPROM/etc implementation variations going forward, and in the meantime this no-op won't break anything new. If the NVRAM source for a device truly isn't OTP, we'll need to add a chip/core quirk, or additional general heuristic, to code determining nvram_src.

1237–1253

This will only apply on "mux-quirked" devices, which are BCM4331 and BCM4360 WiFi (non-SoC) chipsets configured with SPROM -- It will return true immediately for all other devices.

1425–1428

With these changes, D6250 can use bus_set_resource when instantiating children, and allow resource requests to be passed directly to the nexus/rman.

The IRQ situation can be handled by continuing to hardcode the IRQs (in the subsidiary drivers); we can improve that in the future by performing some degree of IRQ RID mapping in a local bus_alloc_resource that delegates to bus_generic_rl_alloc_resource.

In the case of WiFi chipsets, we need to delegate resource APIs to the bridge, and not a local rman, since the bridge is the only device with a view of the available register window mappings.

sys/dev/bhnd/cores/chipc/chipcreg.h

125–128

This is it's previous value, I just moved here down here. I needed it large enough to cover CHIPC_EROMPTR+sizeof(uint32_t).

[landon_landonf.org]

Added BCM43602 note.

sys/dev/bhnd/cores/chipc/chipc.c

1237–1253

Forgot to include BCM43602, which is also a PCIe WiFi device. It's a FullMAC chipset with an attached ARM core, but unlike SoCs, IIRC ChipCommon should be brought up before the ARM core's FullMAC-specific CPU driver.

[landon_landonf.org]

Based on IRC conversation, the plan here is:

1. Adjust CC's allocation to only claim 0x0+0x30
2. Integrate chipc_capabilities struct
3. Integrate resource mapping tables for additional SoC devices (using BUS_ADD_DEVICE + bus_set_resource()).

That should allow D6250's UART/SPI/etc drivers to just drop right in.

[landon_landonf.org]

Implement port-based management/proxying of bhnd_chipc SYS_RES_MEMORY resource mappings.

This is a work-around for RF_SHAREABLE's SYS_RES_MEMORY limitations to allow us to freely
share memory resources that are allocated both by bhnd_chipc, and its attached children,
as is necessary when granting children access to a subset of the chipc device register
space.

I implemented reference counting of allocation/activation requests, which chipc uses
to decide when to allocate/activate/deactivate/release resources from the parent bhnd(4)
bus.

Devices underneath chipc are granted newly allocated resources pointing to (possibly a
subregion of) the resources allocated by chipc.

I modeled this approach on vga_pci.c, which implements similar bridging (of PCI BAR resources)
for its children.

Non-SYS_RES_MEMORY resources are not managed by the chipc driver, since RF_SHAREABLE works
just fine with IRQs.

[landon_landonf.org]

Drop extraneous changes to bhndb that break the i386 build.

[landon_landonf.org]

When vending siba(4) region addresses, ensure that the port address range matches the resource defaults.

This fixes a bhnd_chipc attach issue I'd missed on siba(4) devices.

[landon_landonf.org]

Fix a typo helpfully caught by gcc/tinderbox.

This worked due to the test condition not actually being hit in existing configurations.

[landon_landonf.org]

Re-integration of chipc_cap handling

Re-introduction of chipc_cap handling based on Michael Zhilin's code, along with some minor improvements:

• Adds support for determining a supported flash type based on CAP/EXT_CAP and device-specific quirks
• Exposes CFI bus width via chipc_caps for use by attached cfi device.
• Fetches additional capability info from the EXT_CAP and OTPLAYOUT registers
• Allows chipc children to allocate arbitrary memory ranges when they fall outside of chipc's shared port regions.

[landon_landonf.org]

Based on feedback, lift a number of chipc subroutines out into chipc_subr.c

[mizhka]

my 5 cents:

• I can't apply patch to HEAD. may be my issue, so ship it :)
• bus_adjust_resource man specifies that new range should overlap old range. No sure, if it's recommendation or strong requirement.