Page MenuHomeFreeBSD
Differential D55315

powerpc: add in some busdma domain setup
Needs ReviewPublic
Actions

Authored by adrian on Mon, Feb 16, 8:11 PM.
Tags
Referenced Files
Unknown Object (File)
Fri, Feb 20, 3:14 PM
Unknown Object (File)
Thu, Feb 19, 4:28 AM
Unknown Object (File)
Thu, Feb 19, 12:23 AM
Unknown Object (File)
Thu, Feb 19, 12:17 AM
Unknown Object (File)
Wed, Feb 18, 4:05 AM
Unknown Object (File)
Wed, Feb 18, 2:06 AM
Unknown Object (File)
Tue, Feb 17, 4:25 PM
Subscribers
alfredo
bdragon
imp
jhibbits
luporl

Details

Reviewers
jhibbits
tpearson_raptorengineering.com
Summary
  • Implement the basic tag domain set routine
  • Set the domain to the parent domain if provided

This is just plumbing for eventual work to re-allocate things into
the currently configured domain.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 70743
Build 67626: arc lint + arc unit

Event Timeline

adrian created this revision.Mon, Feb 16, 8:11 PM
Herald added subscribers: jhibbits, imp. · View Herald TranscriptMon, Feb 16, 8:11 PM
adrian requested review of this revision.Mon, Feb 16, 8:11 PM
Harbormaster completed remote builds in B70715: Diff 172031.Mon, Feb 16, 8:11 PM
adrian added a parent revision: D55314: powerpc: create a tag with the parents implementation if supplied.Mon, Feb 16, 8:12 PM
adrian added reviewers: jhibbits, tpearson_raptorengineering.com.Tue, Feb 17, 4:12 PM
adrian added a project: PowerPC.
Herald added subscribers: alfredo, luporl, bdragon. · View Herald TranscriptTue, Feb 17, 4:12 PM
adrian updated this revision to Diff 172129.Wed, Feb 18, 6:41 AM

refactor

Harbormaster completed remote builds in B70742: Diff 172129.Wed, Feb 18, 6:41 AM
adrian updated this revision to Diff 172130.Wed, Feb 18, 6:43 AM

refactor

Harbormaster completed remote builds in B70743: Diff 172130.Wed, Feb 18, 6:43 AM
adrian added a child revision: D55340: powerpc: refactor common busdma tag setup.Wed, Feb 18, 6:53 AM
tpearson_raptorengineering.com added a comment.Wed, Feb 18, 2:39 PM

Unfortunately this fails to boot on POWER9/PowerNV with the following patch series:

D54745
D55313
D55314
D55315

It looks like DMA is being mismanaged vs. my patches in D55084. The low memory DMA bounce buffers are being consumed by the LSI card, even though attach fails, and the USB driver cannot properly attach, followed by the AHCI driver running out of buffers and stalling.

Waking up CPU 9 (dev=15)
Adding CPU 9, hwref=15, awake=1
Waking up CPU 10 (dev=16)
Adding CPU 10, hwref=16, awake=1
Waking up CPU 11 (dev=17)
Adding CPU 11, hwref=17, awake=1
Waking up CPU 12 (dev=c)
Adding CPU 12, hwref=c, awake=1
Waking up CPU 13 (dev=d)
Adding CPU 13, hwref=d, awake=1
Waking up CPU 14 (dev=e)
Adding CPU 14, hwref=e, awake=1
Waking up CPU 15 (dev=f)
Adding CPU 15, hwref=f, awake=1
SMP: AP CPU #2 launched
SMP: AP CPU #14 launched
SMP: AP CPU #5 launched
SMP: AP CPU #11 launched
SMP: AP CPU #10 launched
SMP: AP CPU #9 launched
SMP: AP CPU #3 launched
SMP: AP CPU #13 launched
SMP: AP CPU #6 launched
SMP: AP CPU #1 launched
SMP: AP CPU #8 launched
SMP: AP CPU #15 launched
SMP: AP CPU #4 launched
SMP: AP CPU #12 launched
SMP: AP CPU #7 launched
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
ada0: Serial Number JK11A8B9H82U0F
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors)
mmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
pass0 at ahcich1 bus 0 scbus1 target 0 lun 0
pass0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
pass0: Serial Nmmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
umber JK11A8B9H82U0F
pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
pass0: Command Queueing enabled
WARNING: WITNESS option enabled, expect reduced performance.
GEOM: new disk ada0
GEOM: ada0: the secondary GPT header is not in the last LBA.
GEOM: diskid/DISK-JK11A8B9H82U0F: the secondary GPT header is not in the last LBA.
opal0: providing initial system time
failed to load CTF for kernel: 2
start_init: trying /sbin/init
uhub0: 8 ports with 8 removable, self powered
Waiting 30s for the root mount holders: usbus0..............................
Setting hostuuid: ce1c3c60-3bec-48a7-8c41-adc4cb68bbb1.
Setting hostid: 0xfc69d1ea.
Starting file system checks:
/dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ada0p2: clean, 99494815 free (82047 frags, 12426596 blocks, 0.1% fragmentation)
/dev/ada0p1: FILESYSTEM CLEAN; SKIPPING CHECKS
Mounting local filesystems:.

root@blackbird:~# obmcutil chassisoff
root@blackbird:~# obmcutil poweron
obmc-root@blackbird:~# obmc-console-client


--== Welcome to Hostboot hostboot-a98d734/hbicore.bin ==--

  3.07021|secure|SecureROM valid - enabling functionality
  9.09522|Booting from SBE side 0 on master proc=00050000
  9.13998|ISTEP  6. 5 - host_init_fsi
  9.43559|ISTEP  6. 6 - host_set_ipl_parms
  9.93254|ISTEP  6. 7 - host_discover_targets
 10.73111|HWAS|PRESENT> DIMM[03]=0080000000000000
 10.73113|HWAS|PRESENT> Proc[05]=8000000000000000
 10.73114|HWAS|PRESENT> Core[07]=5440000000000000
 11.15099|ISTEP  6. 8 - host_update_master_tpm
 11.22898|SECURE|Security Access Bit> 0x0000000000000000
 11.22899|SECURE|Secure Mode Disable (via Jumper)> 0x8000000000000000
 11.39930|ISTEP  6. 9 - host_gard
 12.22854|HWAS|FUNCTIONAL> DIMM[03]=0080000000000000
 12.22855|HWAS|FUNCTIONAL> Proc[05]=8000000000000000
 12.22857|HWAS|FUNCTIONAL> Core[07]=5440000000000000
 12.23109|ISTEP  6.11 - host_start_occ_xstop_handler
 13.53761|ISTEP  6.12 - host_voltage_config
 13.82064|ISTEP  7. 1 - mss_attr_cleanup
 15.17491|ISTEP  7. 2 - mss_volt
 15.58481|ISTEP  7. 3 - mss_freq
 16.37109|ISTEP  7. 4 - mss_eff_config
 17.54065|ISTEP  7. 5 - mss_attr_update
 17.60500|ISTEP  8. 1 - host_slave_sbe_config
 18.20697|ISTEP  8. 2 - host_setup_sbe
 18.20816|ISTEP  8. 3 - host_cbs_start
 18.20933|ISTEP  8. 4 - proc_check_slave_sbe_seeprom_complete
 18.23619|ISTEP  8. 5 - host_attnlisten_proc
 18.23828|ISTEP  8. 6 - host_p9_fbc_eff_config
 18.24376|ISTEP  8. 7 - host_p9_eff_config_links
 18.26514|ISTEP  8. 8 - proc_attr_update
 18.26635|ISTEP  8. 9 - proc_chiplet_fabric_scominit
 18.28503|ISTEP  8.10 - proc_xbus_scominit
 18.30622|ISTEP  8.11 - proc_xbus_enable_ridi
 18.33364|ISTEP  8.12 - host_set_voltages
 18.34990|ISTEP  9. 1 - fabric_erepair
 18.42433|ISTEP  9. 2 - fabric_io_dccal
 18.42981|ISTEP  9. 3 - fabric_pre_trainadv
 18.43460|ISTEP  9. 4 - fabric_io_run_training
 18.43642|ISTEP  9. 5 - fabric_post_trainadv
 18.43821|ISTEP  9. 6 - proc_smp_link_layer
 18.44272|ISTEP  9. 7 - proc_fab_iovalid
 18.82881|ISTEP  9. 8 - host_fbc_eff_config_aggregate
 18.85603|ISTEP 10. 1 - proc_build_smp
 20.58193|ISTEP 10. 2 - host_slave_sbe_update
 22.37205|ISTEP 10. 4 - proc_cen_ref_clk_enable
 22.47185|ISTEP 10. 5 - proc_enable_osclite
 22.47309|ISTEP 10. 6 - proc_chiplet_scominit
 22.54399|ISTEP 10. 7 - proc_abus_scominit
 22.56298|ISTEP 10. 8 - proc_obus_scominit
 22.56479|ISTEP 10. 9 - proc_npu_scominit
 22.58516|ISTEP 10.10 - proc_pcie_scominit
 22.64510|ISTEP 10.11 - proc_scomoverride_chiplets
 22.64819|ISTEP 10.12 - proc_chiplet_enable_ridi
 22.66350|ISTEP 10.13 - host_rng_bist
 22.68473|ISTEP 10.14 - host_update_redundant_tpm
 22.71275|ISTEP 11. 1 - host_prd_hwreconfig
 23.38855|ISTEP 11. 2 - cen_tp_chiplet_init1
 23.39225|ISTEP 11. 3 - cen_pll_initf
 23.40985|ISTEP 11. 4 - cen_pll_setup
 23.41386|ISTEP 11. 5 - cen_tp_chiplet_init2
 23.42906|ISTEP 11. 6 - cen_tp_arrayinit
 23.43263|ISTEP 11. 7 - cen_tp_chiplet_init3
 23.43695|ISTEP 11. 8 - cen_chiplet_init
 23.44096|ISTEP 11. 9 - cen_arrayinit
 23.44498|ISTEP 11.10 - cen_initf
 23.45045|ISTEP 11.11 - cen_do_manual_inits
 23.45412|ISTEP 11.12 - cen_startclocks
 23.45809|ISTEP 11.13 - cen_scominits
 23.46208|ISTEP 12. 1 - mss_getecid
 25.31599|ISTEP 12. 2 - dmi_attr_update
 25.35407|ISTEP 12. 3 - proc_dmi_scominit
 25.39113|ISTEP 12. 4 - cen_dmi_scominit
 25.39488|ISTEP 12. 5 - dmi_erepair
 25.54720|ISTEP 12. 6 - dmi_io_dccal
 25.55195|ISTEP 12. 7 - dmi_pre_trainadv
 25.55616|ISTEP 12. 8 - dmi_io_run_training
 25.64106|ISTEP 12. 9 - dmi_post_trainadv
 25.64571|ISTEP 12.10 - proc_cen_framelock
 25.64938|ISTEP 12.11 - host_startprd_dmi
 25.65334|ISTEP 12.12 - host_attnlisten_memb
 25.65747|ISTEP 12.13 - cen_set_inband_addr
 25.66147|ISTEP 13. 1 - host_disable_memvolt
 26.33969|ISTEP 13. 2 - mem_pll_reset
 26.40041|ISTEP 13. 3 - mem_pll_initf
 26.42988|ISTEP 13. 4 - mem_pll_setup
 26.44468|ISTEP 13. 6 - mem_startclocks
 26.48124|ISTEP 13. 7 - host_enable_memvolt
 26.49083|ISTEP 13. 8 - mss_scominit
 28.05078|ISTEP 13. 9 - mss_ddr_phy_reset
 28.46080|ISTEP 13.10 - mss_draminit
 28.69668|ISTEP 13.11 - mss_draminit_training
 29.88436|ISTEP 13.12 - mss_draminit_trainadv
 29.92336|ISTEP 13.13 - mss_draminit_mc
 29.99860|ISTEP 14. 1 - mss_memdiag
 36.33139|ISTEP 14. 2 - mss_thermal_init
 36.37265|ISTEP 14. 3 - proc_pcie_config
 36.42836|ISTEP 14. 4 - mss_power_cleanup
 36.43174|ISTEP 14. 5 - proc_setup_bars
 36.46345|ISTEP 14. 6 - proc_htm_setup
 36.47415|ISTEP 14. 7 - proc_exit_cache_contained
 36.70876|ISTEP 15. 1 - host_build_stop_image
 39.79323|ISTEP 15. 2 - proc_set_pba_homer_bar
 39.87141|ISTEP 15. 3 - host_establish_ex_chiplet
 39.89911|ISTEP 15. 4 - host_start_stop_engine
 39.93454|ISTEP 16. 1 - host_activate_master
 41.19168|ISTEP 16. 2 - host_activate_slave_cores
 41.26785|ISTEP 16. 3 - host_secure_rng
 41.27515|ISTEP 16. 4 - mss_scrub
 41.32749|ISTEP 16. 5 - host_load_io_ppe
 41.34395|ISTEP 16. 6 - host_ipl_complete
 42.09036|ISTEP 18.11 - proc_tod_setup
 42.35223|ISTEP 18.12 - proc_tod_init
 42.41859|ISTEP 20. 1 - host_load_payload
 43.27154|ISTEP 20. 2 - host_load_hdat
 44.15181|ISTEP 21. 1 - host_runtime_setup
 50.02475|htmgt|OCCs are now running in ACTIVE state
 53.41526|ISTEP 21. 2 - host_verify_hdat
 53.46869|ISTEP 21. 3 - host_start_payload
[   54.274605611,5] OPAL skiboot-ecb1dc7 starting...
[   54.274609163,7] initial console log level: memory 7, driver 5
[   54.274611584,6] CPU: P9 generation processor (max 4 threads/core)
[   54.274613762,7] CPU: Boot CPU PIR is 0x000c PVR is 0x004e1203
[   54.274616775,7] OPAL table: 0x3018b160 .. 0x3018b700, branch table: 0x30120000
[   54.274620403,7] Assigning physical memory map table for nimbus
[   54.274623366,7] Parsing HDAT...
[   54.274624910,7] SPIRA-S found.
[   54.274627518,6] BMC #0: HW version 3, SW version 2, chip DD1.0
[   54.274702479,6] SP Family is ibm,ast2500,openbmc
[   54.274708706,7] LPC: IOPATH chip id = 0
[   54.274710311,7] LPC: FW BAR       = f0000000
[   54.274712231,7] LPC: MEM BAR      = e0000000
[   54.274714025,7] LPC: IO BAR       = d0010000
[   54.274715669,7] LPC: Internal BAR = c0012000
[   54.274728478,7] LPC UART: base addr = 3f8 (3f8) size = 1 clk = 1843200, baud = 115200
[   54.274731634,7] LPC: BT [0, 0] sms_int: 0, bmc_int: 0
[   54.275819718,5] HDAT I2C: found e3p0 - unknown@18 dp:ff (ff:)
[   54.275945368,5] HDAT I2C: found e3p1 - unknown@1c dp:ff (ff:)
[   54.276717136,5] CHIP: Chip ID 0000 type: P9N DD2.30
[   54.277111657,5] PLAT: Detected Blackbird platform
SIGTERM received, booting...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
---<<BOOT>>---
Copyright (c) 1992-2026 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 16.0-CURRENT #0 main-n283987-77a4c45cdeda: Wed Feb 18 08:20:45 CST 2026
    root@freebsd-ppc64le-dev:/usr/obj/usr/src.upstream_testing/powerpc.powerpc64le/sys/GENERIC64LE powerpc
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
Preloaded elf kernel "kernel" at 0xc000000004605040.
cpu0: IBM POWER9 revision 2.3, 2700.00 MHz
cpu0: Features dc007182<PPC32,PPC64,ALTIVEC,FPU,MMU,SMT,ISNOOP,ARCH205,ARCH206,VSX,TRUELE>
cpu0: Features2 bee00000<ARCH207,DSCR,ISEL,TAR,VCRYPTO,ARCH300,IEEE128,DARN>
real memory  = 33997053952 (32422 MB)
available KVA = 34359619583 (32767 MB)
Physical memory chunk(s):
0x0000000000003000 - 0x0000000000002fff, 0 bytes (0 pages)
0x000000000000e000 - 0x000000000000ffff, 8192 bytes (2 pages)
0x0000000000094000 - 0x0000000000ffffff, 16171008 bytes (3948 pages)
0x0000000002000000 - 0x000000000284ffff, 8716288 bytes (2128 pages)
0x0000000004636000 - 0x000000002fffffff, 731684864 bytes (178634 pages)
0x0000000034040000 - 0x00000000efffffff, 3153854464 bytes (769984 pages)
0x0000000100000000 - 0x00000007982edfff, 28323012608 bytes (6914798 pages)
0x00000007cc20a000 - 0x00000007fc72dfff, 810696704 bytes (197924 pages)
0x00000007fdc00000 - 0x00000007ff79ffff, 28966912 bytes (7072 pages)
0x00000007ff7d2000 - 0x00000007ff7effff, 122880 bytes (30 pages)
avail memory = 32984395776 (31456 MB)
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
cpu0: dev=4 domain=0  (BSP)
cpu1: dev=5 domain=0
cpu2: dev=6 domain=0
cpu3: dev=7 domain=0
cpu4: dev=24 domain=0
cpu5: dev=25 domain=0
cpu6: dev=26 domain=0
cpu7: dev=27 domain=0
cpu8: dev=14 domain=0
cpu9: dev=15 domain=0
cpu10: dev=16 domain=0
cpu11: dev=17 domain=0
cpu12: dev=c domain=0
cpu13: dev=d domain=0
cpu14: dev=e domain=0
cpu15: dev=f domain=0
Installed OPAL HMI handler.
random: registering fast source PowerISA DARN random number generator
random: fast provider: "PowerISA DARN random number generator"
random: no preloaded entropy cache
random: no platform bootloader entropy
arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled.
VIMAGE (virtualized network stack) enabled
hostuuid: using 00000000-0000-0000-0000-000000000000
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
ULE: setup cpu 7
ULE: setup cpu 8
ULE: setup cpu 9
ULE: setup cpu 10
ULE: setup cpu 11
ULE: setup cpu 12
ULE: setup cpu 13
ULE: setup cpu 14
ULE: setup cpu 15
firmware: 't4fw_cfg' version 0: 4523 bytes loaded at 0xc0000000039bac9c
firmware: 't4fw_cfg_uwire' version 0: 21433 bytes loaded at 0xc0000000039bbe47
firmware: 't4fw' version 0: 570880 bytes loaded at 0xc0000000039c1200
firmware: 't5fw_cfg' version 0: 5902 bytes loaded at 0xc000000003a4c800
firmware: 't5fw_cfg_uwire' version 0: 21859 bytes loaded at 0xc000000003a4df0e
firmware: 't5fw' version 0: 678912 bytes loaded at 0xc000000003a53471
firmware: 't6fw_cfg' version 0: 5960 bytes loaded at 0xc000000003af9071
firmware: 't6fw_cfg_uwire' version 0: 22693 bytes loaded at 0xc000000003afa7b9
firmware: 't6fw' version 0: 730624 bytes loaded at 0xc000000003b0005e
firmware: 't7fw_cfg' version 0: 24216 bytes loaded at 0xc000000003bb265e
firmware: 't7fw_cfg_uwire' version 0: 24212 bytes loaded at 0xc000000003bb84f6
random: entropy device external interface
wlan: <802.11 Link Layer>
firmware: 'ice_ddp' version 0: 1352580 bytes loaded at 0xc000000003dc23b2
firmware: 'isp_2400' version 1: 204396 bytes loaded at 0xc000000003bc4548
firmware: 'isp_2500' version 1: 205736 bytes loaded at 0xc000000003c0519c
firmware: 'isp_2600' version 1: 290876 bytes loaded at 0xc000000003c43c68
firmware: 'isp_2700' version 1: 343552 bytes loaded at 0xc000000003c9b0f0
firmware: 'isp_2800' version 1: 380416 bytes loaded at 0xc000000003d060f0
[ath_hal] loaded
kbd0 at kbdmux0
mem: <memory>
null: <full device, null device, zero device>
openfirm: <Open Firmware control device>
sound_global_init: snd_unit=-1 snd_vchans_enable=1 latency=2 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25
tcp_log: tcp_log device
crypto: <crypto core>
ofwbus0: <Open Firmware Device Tree> on nexus0
opal0: <OPAL Abstraction Firmware> irq 1048560,1048561,1048562,1048563,1048564,1048565,1048567,1048568,1048569,1048570,1048571,1048572,1048573,1044478,1044479,1040382,1040383,1038334,1038335,1036286,1036287,1032190,1032191,1030142,1030143 on ofwbus0
opal0: registered as a time-of-day clock, resolution 0.002000s
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
ofwbus0: no default resources for rid = 25, type = 1
simplebus0: <Flattened device tree simple bus> mem 0x6030000000000-0x60300ffffffff on ofwbus0
pcib0: <OPAL Host-PCI bridge> mem 0x600c3c0000000-0x600c3c0000fff,0x600c300000000-0x600c30fffffff on ofwbus0
pcib0: OPAL ID 0 reserved PE# 1ff
mmu_radix_mapdev_attr(0x600c3c0000000, 4096, 0)
pcib0: Enabled segmented 64-bit memory window #31 at 0x6000000000000 size 0x4000000000 segment size 0x20000000
pcib0: Supports 4088 MSIs starting at 1040384
pci0: <OFW PCI bus> numa-domain 0 on pcib0
pci0: domain=0, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=0, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=34
pcib1: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci0
pcib1:   domain            0
pcib1:   secondary bus     1
pcib1:   subordinate bus   34
pcib1:   memory decode     0x80000000-0xffefffff
pcib1:   prefetched decode 0x6000000000000-0x6003fbfffffff
pci1: <OFW PCI bus> numa-domain 0 on pcib1
pcib1: allocated bus range (1-1) for rid 0 of pci1
pci1: domain=0, physical bus=1
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=1, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=2, subbus=34
pcib1: allocated bus range (2-34) for rid 0 of pci0:1:0:0
pcib2: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci1
pcib1: allocated memory range (0x80000000-0xffefffff) for rid 20 of pcib2
pcib1: allocated prefetch range (0x6000000000000-0x6003fbfffffff) for rid 24 of pcib2
pcib2:   domain            0
pcib2:   secondary bus     2
pcib2:   subordinate bus   34
pcib2:   memory decode     0x80000000-0xffefffff
pcib2:   prefetched decode 0x6000000000000-0x6003fbfffffff
pci2: <OFW PCI bus> numa-domain 0 on pcib2
pcib2: allocated bus range (2-2) for rid 0 of pci2
pci2: domain=0, physical bus=2
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=3, subbus=3
pcib2: allocated bus range (3-3) for rid 0 of pci0:2:0:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=1, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=4, subbus=4
pcib2: allocated bus range (4-4) for rid 0 of pci0:2:1:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=2, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=5, subbus=9
pcib2: allocated bus range (5-9) for rid 0 of pci0:2:2:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=3, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=10, subbus=14
pcib2: allocated bus range (10-14) for rid 0 of pci0:2:3:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=4, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=15, subbus=19
pcib2: allocated bus range (15-19) for rid 0 of pci0:2:4:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=5, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=20, subbus=24
pcib2: allocated bus range (20-24) for rid 0 of pci0:2:5:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=6, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=25, subbus=29
pcib2: allocated bus range (25-29) for rid 0 of pci0:2:6:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=7, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=30, subbus=34
pcib2: allocated bus range (30-34) for rid 0 of pci0:2:7:0
pcib3: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80000000-0x803fffff) for rid 20 of pcib3
pcib2: allocated prefetch range (0x6000000000000-0x600001fffffff) for rid 24 of pcib3
pcib3: attempting to allocate 1 MSI vectors (8 supported)
pcib3: using IRQ 34594816 for MSI
pcib3: HotPlug command: 03c0 -> 17eb
pcib3:   domain            0
pcib3:   secondary bus     3
pcib3:   subordinate bus   3
pcib3:   memory decode     0x80000000-0x803fffff
pcib3:   prefetched decode 0x6000000000000-0x600001fffffff
pcib4: <OFW PCI-PCI bridge> at device 1.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80400000-0x807fffff) for rid 20 of pcib4
pcib2: allocated prefetch range (0x6000020000000-0x600003fffffff) for rid 24 of pcib4
pcib4: attempting to allocate 1 MSI vectors (8 supported)
pcib4: using IRQ 34594817 for MSI
pcib4: HotPlug command: 03c0 -> 17eb
pcib4:   domain            0
pcib4:   secondary bus     4
pcib4:   subordinate bus   4
pcib4:   memory decode     0x80400000-0x807fffff
pcib4:   prefetched decode 0x6000020000000-0x600003fffffff
pcib5: <OFW PCI-PCI bridge> at device 2.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80800000-0x80bfffff) for rid 20 of pcib5
pcib2: allocated prefetch range (0x6000040000000-0x600005fffffff) for rid 24 of pcib5
pcib5: attempting to allocate 1 MSI vectors (8 supported)
pcib5: using IRQ 34594818 for MSI
pcib5: HotPlug command: 03c0 -> 17eb
pcib5:   domain            0
pcib5:   secondary bus     5
pcib5:   subordinate bus   9
pcib5:   memory decode     0x80800000-0x80bfffff
pcib5:   prefetched decode 0x6000040000000-0x600005fffffff
pcib6: <OFW PCI-PCI bridge> at device 3.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80c00000-0x80ffffff) for rid 20 of pcib6
pcib2: allocated prefetch range (0x6000060000000-0x600007fffffff) for rid 24 of pcib6
pcib6: attempting to allocate 1 MSI vectors (8 supported)
pcib6: using IRQ 34594819 for MSI
pcib6: HotPlug command: 03c0 -> 17eb
pcib6:   domain            0
pcib6:   secondary bus     10
pcib6:   subordinate bus   14
pcib6:   memory decode     0x80c00000-0x80ffffff
pcib6:   prefetched decode 0x6000060000000-0x600007fffffff
pcib7: <OFW PCI-PCI bridge> at device 4.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81000000-0x813fffff) for rid 20 of pcib7
pcib2: allocated prefetch range (0x6000080000000-0x600009fffffff) for rid 24 of pcib7
pcib7: attempting to allocate 1 MSI vectors (8 supported)
pcib7: using IRQ 34594820 for MSI
pcib7: HotPlug command: 03c0 -> 17eb
pcib7:   domain            0
pcib7:   secondary bus     15
pcib7:   subordinate bus   19
pcib7:   memory decode     0x81000000-0x813fffff
pcib7:   prefetched decode 0x6000080000000-0x600009fffffff
pcib8: <OFW PCI-PCI bridge> at device 5.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81400000-0x817fffff) for rid 20 of pcib8
pcib2: allocated prefetch range (0x60000a0000000-0x60000bfffffff) for rid 24 of pcib8
pcib8: attempting to allocate 1 MSI vectors (8 supported)
pcib8: using IRQ 34594821 for MSI
pcib8: HotPlug command: 03c0 -> 17eb
pcib8:   domain            0
pcib8:   secondary bus     20
pcib8:   subordinate bus   24
pcib8:   memory decode     0x81400000-0x817fffff
pcib8:   prefetched decode 0x60000a0000000-0x60000bfffffff
pcib9: <OFW PCI-PCI bridge> at device 6.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81800000-0x81bfffff) for rid 20 of pcib9
pcib2: allocated prefetch range (0x60000c0000000-0x60000dfffffff) for rid 24 of pcib9
pcib9: attempting to allocate 1 MSI vectors (8 supported)
pcib9: using IRQ 34594822 for MSI
pcib9: HotPlug command: 03c0 -> 17eb
pcib9:   domain            0
pcib9:   secondary bus     25
pcib9:   subordinate bus   29
pcib9:   memory decode     0x81800000-0x81bfffff
pcib9:   prefetched decode 0x60000c0000000-0x60000dfffffff
pcib10: <OFW PCI-PCI bridge> at device 7.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81c00000-0x81ffffff) for rid 20 of pcib10
pcib2: allocated prefetch range (0x60000e0000000-0x60000ffffffff) for rid 24 of pcib10
pcib10: attempting to allocate 1 MSI vectors (8 supported)
pcib10: using IRQ 34594823 for MSI
pcib10: HotPlug command: 03c0 -> 17eb
pcib10:   domain            0
pcib10:   secondary bus     30
pcib10:   subordinate bus   34
pcib10:   memory decode     0x81c00000-0x81ffffff
pcib10:   prefetched decode 0x60000e0000000-0x60000ffffffff
pcib11: <OPAL Host-PCI bridge> mem 0x600c3c0100000-0x600c3c0100fff,0x600c320000000-0x600c32fffffff on ofwbus0
pcib11: OPAL ID 0x1 reserved PE# ff
mmu_radix_mapdev_attr(0x600c3c0100000, 4096, 0)
pcib11: Enabled segmented 64-bit memory window #15 at 0x6004000000000 size 0x4000000000 segment size 0x40000000
pcib11: Supports 2040 MSIs starting at 1038336
pci3: <OFW PCI bus> numa-domain 0 on pcib11
pci3: domain=11, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=11, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib12: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci3
pcib12:   domain            11
pcib12:   secondary bus     1
pcib12:   subordinate bus   1
pcib12:   memory decode     0x80000000-0xffefffff
pci4: <OFW PCI bus> numa-domain 0 on pcib12
pcib12: allocated bus range (1-1) for rid 0 of pci4
pci4: domain=11, physical bus=1
found-> vendor=0x1000, dev=0x0097, revid=0x02
        domain=11, bus=1, slot=0, func=0
        class=01-07-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit, vector masks
        MSI-X supports 96 messages in map 0x14
        map[10]: type I/O Port, range 32, base 0, size  8, port disabled
        map[14]: type Memory, range 64, base 0x80140000, size 16, enabled
pcib12: allocated memory range (0x80140000-0x8014ffff) for rid 14 of pci11:1:0:0
        map[1c]: type Memory, range 64, base 0x80100000, size 18, enabled
pcib12: allocated memory range (0x80100000-0x8013ffff) for rid 1c of pci11:1:0:0
pcib13: <OPAL Host-PCI bridge> mem 0x600c3c0200000-0x600c3c0200fff,0x600c340000000-0x600c34fffffff on ofwbus0
pcib13: OPAL ID 0x2 reserved PE# ff
mmu_radix_mapdev_attr(0x600c3c0200000, 4096, 0)
pcib13: Enabled segmented 64-bit memory window #15 at 0x6008000000000 size 0x4000000000 segment size 0x40000000
pcib13: Supports 2040 MSIs starting at 1036288
pci5: <OFW PCI bus> numa-domain 0 on pcib13
pci5: domain=13, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=13, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib14: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci5
pcib14:   domain            13
pcib14:   secondary bus     1
pcib14:   subordinate bus   1
pcib14:   memory decode     0x80000000-0xffefffff
pci6: <OFW PCI bus> numa-domain 0 on pcib14
pcib14: allocated bus range (1-1) for rid 0 of pci6
pci6: domain=13, physical bus=1
found-> vendor=0x1b4b, dev=0x9235, revid=0x11
        domain=13, bus=1, slot=0, func=0
        class=01-06-01, hdrtype=0x00, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 1 message
        map[10]: type I/O Port, range 32, base 0x8000, size  3, enabled
pcib14: failed to allocate initial I/O port window (0x8000-0x8007,0x8)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x8)
pci6: pci13:1:0:0 bar 0x10 failed to allocate
        map[14]: type I/O Port, range 32, base 0x8040, size  2, port disabled
pcib14: failed to allocate initial I/O port window (0x8040-0x8043,0x4)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x4)
pci6: pci13:1:0:0 bar 0x14 failed to allocate
        map[18]: type I/O Port, range 32, base 0x8100, size  3, port disabled
pcib14: failed to allocate initial I/O port window (0x8100-0x8107,0x8)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x8)
pci6: pci13:1:0:0 bar 0x18 failed to allocate
        map[1c]: type I/O Port, range 32, base 0x8140, size  2, port disabled
pcib14: failed to allocate initial I/O port window (0x8140-0x8143,0x4)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x4)
pci6: pci13:1:0:0 bar 0x1c failed to allocate
        map[20]: type I/O Port, range 32, base 0x800000, size  5, port disabled
pcib14: failed to allocate initial I/O port window (0x800000-0x80001f,0x20)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x20)
pci6: pci13:1:0:0 bar 0x20 failed to allocate
        map[24]: type Memory, range 32, base 0x80000000, size 11, enabled
pcib14: allocated memory range (0x80000000-0x800007ff) for rid 24 of pci13:1:0:0
pcib15: <OPAL Host-PCI bridge> mem 0x600c3c0300000-0x600c3c0300fff,0x600c360000000-0x600c36fffffff on ofwbus0
pcib15: OPAL ID 0x3 reserved PE# 1ff
mmu_radix_mapdev_attr(0x600c3c0300000, 4096, 0)
pcib15: Enabled segmented 64-bit memory window #31 at 0x6020000000000 size 0x4000000000 segment size 0x20000000
pcib15: Supports 4088 MSIs starting at 1032192
pci7: <OFW PCI bus> numa-domain 0 on pcib15
pci7: domain=15, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=15, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib16: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci7
pcib16:   domain            15
pcib16:   secondary bus     1
pcib16:   subordinate bus   1
pcib16:   memory decode     0x80000000-0xffefffff
pci8: <OFW PCI bus> numa-domain 0 on pcib16
pcib16: allocated bus range (1-1) for rid 0 of pci8
pci8: domain=15, physical bus=1
found-> vendor=0x104c, dev=0x8241, revid=0x02
        domain=15, bus=1, slot=0, func=0
        class=0c-03-30, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 8 messages in map 0x18
        map[10]: type Memory, range 64, base 0x80000000, size 16, enabled
pcib16: allocated memory range (0x80000000-0x8000ffff) for rid 10 of pci15:1:0:0
        map[18]: type Memory, range 64, base 0x80010000, size 13, enabled
pcib16: allocated memory range (0x80010000-0x80011fff) for rid 18 of pci15:1:0:0
pcib17: <OPAL Host-PCI bridge> mem 0x600c3c0400000-0x600c3c0400fff,0x600c380000000-0x600c38fffffff on ofwbus0
pcib17: OPAL ID 0x4 reserved PE# ff
mmu_radix_mapdev_attr(0x600c3c0400000, 4096, 0)
pcib17: Enabled segmented 64-bit memory window #15 at 0x6024000000000 size 0x4000000000 segment size 0x40000000
pcib17: Supports 2040 MSIs starting at 1030144
pci9: <OFW PCI bus> numa-domain 0 on pcib17
pci9: domain=17, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=17, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib18: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci9
pcib18:   domain            17
pcib18:   secondary bus     1
pcib18:   subordinate bus   1
pcib18:   memory decode     0x80000000-0xffefffff
pcib18:   prefetched decode 0x6024000000000-0x6027f7fffffff
pci10: <OFW PCI bus> numa-domain 0 on pcib18
pcib18: allocated bus range (1-1) for rid 0 of pci10
pci10: domain=17, physical bus=1
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=0
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000000000, size 16, enabled
pcib18: allocated prefetch range (0x6024000000000-0x602400000ffff) for rid 10 of pci17:1:0:0
        map[18]: type Prefetchable Memory, range 64, base 0x6024000010000, size 16, enabled
pcib18: allocated prefetch range (0x6024000010000-0x602400001ffff) for rid 18 of pci17:1:0:0
        map[20]: type Prefetchable Memory, range 64, base 0x6024000020000, size 16, enabled
pcib18: allocated prefetch range (0x6024000020000-0x602400002ffff) for rid 20 of pci17:1:0:0
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=1
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=b, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000030000, size 16, enabled
pcib18: allocated prefetch range (0x6024000030000-0x602400003ffff) for rid 10 of pci17:1:0:1
        map[18]: type Prefetchable Memory, range 64, base 0x6024000040000, size 16, enabled
pcib18: allocated prefetch range (0x6024000040000-0x602400004ffff) for rid 18 of pci17:1:0:1
        map[20]: type Prefetchable Memory, range 64, base 0x6024000050000, size 16, enabled
pcib18: allocated prefetch range (0x6024000050000-0x602400005ffff) for rid 20 of pci17:1:0:1
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=2
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000060000, size 16, enabled
pcib18: allocated prefetch range (0x6024000060000-0x602400006ffff) for rid 10 of pci17:1:0:2
        map[18]: type Prefetchable Memory, range 64, base 0x6024000070000, size 16, enabled
pcib18: allocated prefetch range (0x6024000070000-0x602400007ffff) for rid 18 of pci17:1:0:2
        map[20]: type Prefetchable Memory, range 64, base 0x6024000080000, size 16, enabled
pcib18: allocated prefetch range (0x6024000080000-0x602400008ffff) for rid 20 of pci17:1:0:2
pcib19: <OPAL Host-PCI bridge> mem 0x600c3c0500000-0x600c3c0500fff,0x600c3a0000000-0x600c3afffffff on ofwbus0
pcib19: OPAL ID 0x5 reserved PE# ff
mmu_radix_mapdev_attr(0x600c3c0500000, 4096, 0)
pcib19: Enabled segmented 64-bit memory window #15 at 0x6028000000000 size 0x4000000000 segment size 0x40000000
pcib19: Supports 2040 MSIs starting at 1028096
pci11: <OFW PCI bus> numa-domain 0 on pcib19
pci11: domain=19, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=19, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=2
pcib20: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci11
pcib20:   domain            19
pcib20:   secondary bus     1
pcib20:   subordinate bus   2
pcib20:   memory decode     0x80000000-0xffefffff
pci12: <OFW PCI bus> numa-domain 0 on pcib20
pcib20: allocated bus range (1-1) for rid 0 of pci12
pci12: domain=19, physical bus=1
found-> vendor=0x1a03, dev=0x1150, revid=0x04
        domain=19, bus=1, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0143, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit
        secbus=2, subbus=2
pcib20: allocated bus range (2-2) for rid 0 of pci19:1:0:0
pcib21: <OFW PCI-PCI bridge> irq 1030136 at device 0.0 numa-domain 0 on pci12
pcib20: allocated memory range (0x80000000-0xffefffff) for rid 20 of pcib21
pcib21:   domain            19
pcib21:   secondary bus     2
pcib21:   subordinate bus   2
pcib21:   memory decode     0x80000000-0xffefffff
pci13: <OFW PCI bus> numa-domain 0 on pcib21
pcib21: allocated bus range (2-2) for rid 0 of pci13
pci13: domain=19, physical bus=2
found-> vendor=0x1a03, dev=0x2000, revid=0x41
        domain=19, bus=2, slot=0, func=0
        class=03-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0210, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 2 messages, 64 bit
        map[10]: type Memory, range 32, base 0x80000000, size 24, enabled
pcib21: allocated memory range (0x80000000-0x80ffffff) for rid 10 of pci19:2:0:0
        map[14]: type Memory, range 32, base 0x81000000, size 17, enabled
pcib21: allocated memory range (0x81000000-0x8101ffff) for rid 14 of pci19:2:0:0
        map[18]: type I/O Port, range 32, base 0, size  7, port disabled
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
cpu0: Nominal frequency 2700Mhz
cpu1: <Open Firmware CPU> on cpulist0
cpu1: Nominal frequency 2700Mhz
cpu2: <Open Firmware CPU> on cpulist0
cpu2: Nominal frequency 2700Mhz
cpu3: <Open Firmware CPU> on cpulist0
cpu3: Nominal frequency 2700Mhz
pcib0: Scanning bus (devclass 'pcib')
pci0: Scanning bus (devclass 'pci')
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x1, end 0x22
pcib1: Scanning bus (devclass 'pcib')
pci1: Scanning bus (devclass 'pci')
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x2, end 0x22
pcib2: Scanning bus (devclass 'pcib')
pci2: Scanning bus (devclass 'pci')
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x3, end 0x3
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x4, end 0x4
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x5, end 0x9
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0xa, end 0xe
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0xf, end 0x13
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x14, end 0x18
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x19, end 0x1d
pcib0: Recording child resources
pcib0: Found child resource type 5, start 0x1e, end 0x22
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x1e-0x22]
pcib0: Mapped PE# 1fd to bus
pcib0: Mapping PE# 1fd window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x19-0x1d]
pcib0: Mapped PE# 1fc to bus
pcib0: Mapping PE# 1fc window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x14-0x18]
pcib0: Mapped PE# 1fb to bus
pcib0: Mapping PE# 1fb window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0xf-0x13]
pcib0: Mapped PE# 1fa to bus
pcib0: Mapping PE# 1fa window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0xa-0xe]
pcib0: Mapped PE# 1f9 to bus
pcib0: Mapping PE# 1f9 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 5, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x5-0x9]
pcib0: Mapped PE# 1f8 to bus
pcib0: Mapping PE# 1f8 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Assigning subordinate bus 0x4
pcib0: Mapped PE# 1f7 to bus
pcib0: Mapping PE# 1f7 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Assigning subordinate bus 0x3
pcib0: Mapped PE# 1f6 to bus
pcib0: Mapping PE# 1f6 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 33, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x2-0x22]
pcib0: Mapped PE# 1f5 to bus
pcib0: Mapping PE# 1f5 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib0: Invalid subordinate bus count 34, defaulting to exact bus match
pcib0: Assigning subordinate bus range [0x1-0x22]
pcib0: Mapped PE# 1f4 to bus
pcib0: Mapping PE# 1f4 window #0 0-0x1000 TCE segment size 40000000
pcib0: Enabling 64-bit DMA bypass mode
pcib11: Scanning bus (devclass 'pcib')
pci3: Scanning bus (devclass 'pci')
pcib11: Recording child resources
pcib11: Found child resource type 5, start 0x1, end 0x1
pcib12: Scanning bus (devclass 'pcib')
pci4: Scanning bus (devclass 'pci')
pcib11: Recording child resources
pcib11: Found child resource type 3, start 0x80140000, end 0x8014ffff
pcib11: Found child resource type 3, start 0x80100000, end 0x8013ffff
pcib11: Assigning subordinate bus 0x1
pcib11: Mapped PE# fd to bus
pcib11: Mapping PE# fd window #0 0-0x1000 TCE segment size 40000000
pcib11: Enabling 64-bit DMA bypass mode
pcib11: Enabled 32-bit memory window on PE# fd at 0x80100000 size 0x3ffff
pcib11: Enabled 32-bit memory window on PE# fd at 0x80140000 size 0xffff
pcib13: Scanning bus (devclass 'pcib')
pci5: Scanning bus (devclass 'pci')
pcib13: Recording child resources
pcib13: Found child resource type 5, start 0x1, end 0x1
pcib14: Scanning bus (devclass 'pcib')
pci6: Scanning bus (devclass 'pci')
pcib13: Recording child resources
pcib13: Found child resource type 3, start 0x80000000, end 0x800007ff
pcib13: Assigning subordinate bus 0x1
pcib13: Mapped PE# fd to bus
pcib13: Mapping PE# fd window #0 0-0x1000 TCE segment size 40000000
pcib13: Enabling 64-bit DMA bypass mode
pcib13: Enabled 32-bit memory window on PE# fd at 0x80000000 size 0x7ff
pcib15: Scanning bus (devclass 'pcib')
pci7: Scanning bus (devclass 'pci')
pcib15: Recording child resources
pcib15: Found child resource type 5, start 0x1, end 0x1
pcib16: Scanning bus (devclass 'pcib')
pci8: Scanning bus (devclass 'pci')
pcib15: Recording child resources
pcib15: Found child resource type 3, start 0x80000000, end 0x8000ffff
pcib15: Found child resource type 3, start 0x80010000, end 0x80011fff
pcib15: Assigning subordinate bus 0x1
pcib15: Mapped PE# 1fd to bus
pcib15: Mapping PE# 1fd window #0 0-0x1000 TCE segment size 40000000
pcib15: Enabling 64-bit DMA bypass mode
pcib15: Enabled 32-bit memory window on PE# 1fd at 0x80010000 size 0x1fff
pcib15: Enabled 32-bit memory window on PE# 1fd at 0x80000000 size 0xffff
pcib17: Scanning bus (devclass 'pcib')
pci9: Scanning bus (devclass 'pci')
pcib17: Recording child resources
pcib17: Found child resource type 5, start 0x1, end 0x1
pcib18: Scanning bus (devclass 'pcib')
pci10: Scanning bus (devclass 'pci')
pcib17: Recording child resources
pcib17: Found child resource type 3, start 0x6024000000000, end 0x602400000ffff
pcib17: Found child resource type 3, start 0x6024000010000, end 0x602400001ffff
pcib17: Found child resource type 3, start 0x6024000020000, end 0x602400002ffff
pcib17: Recording child resources
pcib17: Found child resource type 3, start 0x6024000030000, end 0x602400003ffff
pcib17: Found child resource type 3, start 0x6024000040000, end 0x602400004ffff
pcib17: Found child resource type 3, start 0x6024000050000, end 0x602400005ffff
pcib17: Recording child resources
pcib17: Found child resource type 3, start 0x6024000060000, end 0x602400006ffff
pcib17: Found child resource type 3, start 0x6024000070000, end 0x602400007ffff
pcib17: Found child resource type 3, start 0x6024000080000, end 0x602400008ffff
pcib17: Overriding PE# fd to PE# 0 due to 64-bit segmented memory constraint
pcib17: Assigning subordinate bus 0x1
pcib17: Mapped PE# 0 to bus
pcib17: Mapping PE# 0 window #0 0-0x1000 TCE segment size 40000000
pcib17: Enabling 64-bit DMA bypass mode
pcib19: Scanning bus (devclass 'pcib')
pci11: Scanning bus (devclass 'pci')
pcib19: Recording child resources
pcib19: Found child resource type 5, start 0x1, end 0x2
pcib20: Scanning bus (devclass 'pcib')
pci12: Scanning bus (devclass 'pci')
pcib19: Recording child resources
pcib19: Found child resource type 5, start 0x2, end 0x2
pcib21: Scanning bus (devclass 'pcib')
pci13: Scanning bus (devclass 'pci')
pcib19: Recording child resources
pcib19: Found child resource type 3, start 0x80000000, end 0x80ffffff
pcib19: Found child resource type 3, start 0x81000000, end 0x8101ffff
pcib19: Assigning subordinate bus 0x2
pcib19: Mapped PE# fd to bus
pcib19: Mapping PE# fd window #0 0-0x1000 TCE segment size 40000000
pcib19: Enabling 64-bit DMA bypass mode
pcib19: Assigning subordinate bus range [0x1-0x2]
pcib19: Mapped PE# fc to bus
pcib19: Mapping PE# fc window #0 0-0x1000 TCE segment size 40000000
pcib19: Enabling 64-bit DMA bypass mode
pcib19: Enabled 32-bit memory window on PE# fd at 0x81000000 size 0x1ffff
pcib19: Enabled 32-bit memory window on PE# fd at 0x80000000 size 0xffffff
xive0: <External Interrupt Virtualization Engine> mem 0x6030203180000-0x603020318ffff,0x6030203190000-0x603020319ffff,0x60302031a0000-0x60302031affff,0x60302031b0000-0x60302031bffff on ofwbus0
mmu_radix_mapdev_attr(0x6030203190000, 65536, 0)
mmu_radix_mapdev_attr(0x6010000410000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000400000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000430000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000420000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000450000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000440000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000470000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000460000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000490000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000480000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004b0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004a0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004d0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004c0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004f0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004e0000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000510000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000500000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000530000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000520000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000550000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000540000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000570000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000560000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000590000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000580000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005b0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005a0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005d0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005c0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005f0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005e0000, 131072, 0)
xivevc0: <External Interrupt Virtualization Engine Root> on ofwbus0
pmcr0: <Power Management Control Register> on cpu0
cpufreq0: <CPU frequency control> on cpu0
pmcr1: <Power Management Control Register> on cpu1
cpufreq1: <CPU frequency control> on cpu1
pmcr2: <Power Management Control Register> on cpu2
cpufreq2: <CPU frequency control> on cpu2
pmcr3: <Power Management Control Register> on cpu3
cpufreq3: <CPU frequency control> on cpu3
ofwbus0: <bmc> compat ibm,ast2500,openbmc (no driver attached)
opalcons0: <OPAL Consoles> on opal0
uart0: <OPAL Serial Port> on opalcons0
uart0: console
opalflash0: <OPAL System Flash> on opal0
opalsens0: <OPAL Sensors> on opal0
opal_sensor0: <OPAL sensor> on opalsens0
opal_sensor1: <OPAL sensor> on opalsens0
opal_sensor2: <OPAL sensor> on opalsens0
opal_sensor3: <OPAL sensor> on opalsens0
opal_sensor4: <OPAL sensor> on opalsens0
opal_sensor5: <OPAL sensor> on opalsens0
opal_sensor6: <OPAL sensor> on opalsens0
opal_sensor7: <OPAL sensor> on opalsens0
opal_sensor8: <OPAL sensor> on opalsens0
opal_sensor9: <OPAL sensor> on opalsens0
opal_sensor10: <OPAL sensor> on opalsens0
opal_sensor11: <OPAL sensor> on opalsens0
opal_sensor12: <OPAL sensor> on opalsens0
opal_sensor13: <OPAL sensor> on opalsens0
opal_sensor14: <OPAL sensor> on opalsens0
opal_sensor15: <OPAL sensor> on opalsens0
opal_sensor16: <OPAL sensor> on opalsens0
opal_sensor17: <OPAL sensor> on opalsens0
opal_sensor18: <OPAL sensor> on opalsens0
opal_sensor19: <OPAL sensor> on opalsens0
opal_sensor20: <OPAL sensor> on opalsens0
opal_sensor21: <OPAL sensor> on opalsens0
opal_sensor22: <OPAL sensor> on opalsens0
opal_sensor23: <OPAL sensor> on opalsens0
opal_sensor24: <OPAL sensor> on opalsens0
opal_sensor25: <OPAL sensor> on opalsens0
opal_sensor26: <OPAL sensor> on opalsens0
opal_sensor27: <OPAL sensor> on opalsens0
opal_sensor28: <OPAL sensor> on opalsens0
opal_sensor29: <OPAL sensor> on opalsens0
opal_sensor30: <OPAL sensor> on opalsens0
opal_sensor31: <OPAL sensor> on opalsens0
opal_sensor32: <OPAL sensor> on opalsens0
opal_sensor33: <OPAL sensor> on opalsens0
ofwbus0: <ibm,secureboot> compat ibm,secureboot (no driver attached)
ofwbus0: <imc-counters> compat ibm,opal-in-memory-counters (no driver attached)
simplebus0: <lpc-controller@c0012000> mem 0xc0012000-0xc00120ff compat ibm,power9-lpc-controller (no driver attached)
simplebus0: <lpc@0> compat ibm,power9-lpc (no driver attached)
simplebus0: <opb-arbiter@c0011000> mem 0xc0011000-0xc0011007 compat ibm,power9-lpcm-opb-arbiter (no driver attached)
simplebus0: <opb-master@c0010000> mem 0xc0010000-0xc001005f compat ibm,power9-lpcm-opb-master (no driver attached)
mpr0: <Avago Technologies (LSI) SAS3008> mem 0x80140000-0x8014ffff,0x80100000-0x8013ffff irq 1040376 at device 0.0 numa-domain 0 on pci4
pci4: child mpr0 requested type 3 for rid 0x10, but the BAR says it is an ioport
ofw_pci mapdev: start 600c080140000, len 65536
mmu_radix_mapdev_attr(0x600c080140000, 65536, 0)
mpr0: attempting to allocate 1 MSI-X vectors (96 supported)
mpr0: using IRQ 35637240 for MSI-X
mpr0: Firmware: 16.00.01.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
mpr0: IOC Facts allocation failed with error 12
device_attach: mpr0 attach returned 12
ahci0: <Marvell 88SE9235 AHCI SATA controller> mem 0x80000000-0x800007ff irq 1038328 at device 0.0 numa-domain 0 on pci6
ofw_pci mapdev: start 600c100000000, len 2048
mmu_radix_mapdev_attr(0x600c100000000, 4096, 0)
ahci0: attempting to allocate 1 MSI vectors (1 supported)
ahci0: using IRQ 36675568 for MSI
ahci0: AHCI v1.00 with 4 6Gbps ports, Port Multiplier supported with FBS
ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 4ports
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: Caps: CPD ESP FBSCP
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: Caps: CPD ESP FBSCP
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich2: Caps: CPD ESP FBSCP
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich3: Caps: CPD ESP FBSCP
xhci0: <XHCI (generic) USB 3.0 controller> mem 0x80000000-0x8000ffff,0x80010000-0x80011fff irq 1036280 at device 0.0 numa-domain 0 on pci8
ofw_pci mapdev: start 600c180000000, len 65536
mmu_radix_mapdev_attr(0x600c180000000, 65536, 0)
xhci0: 64 bytes context size, 64-bit DMA
xhci0: xECP capabilities <LEGACY,PROTO,PROTO>
ofw_pci mapdev: start 600c180010000, len 8192
mmu_radix_mapdev_attr(0x600c180010000, 8192, 0)
xhci0: attempting to allocate 1 MSI-X vectors (8 supported)
xhci0: using IRQ 37709800 for MSI-X
xhci0: MSI-X enabled
xhci0: (New XHCI DeviceId=0x8241104c)
usbus0 numa-domain 0 on xhci0
xhci0: usbpf: Attached
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000000000-0x602400000ffff,0x6024000010000-0x602400001ffff,0x6024000020000-0x602400002ffff irq 1032184 at device 0.0 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000000000, len 65536
mmu_radix_mapdev_attr(0x6024000000000, 65536, 0)
ofw_pci mapdev: start 6024000010000, len 65536
mmu_radix_mapdev_attr(0x6024000010000, 65536, 0)
bge0: APE FW version: NCSI v0.6.0.18
bge0: attempting to allocate 1 MSI vectors (8 supported)
bge0: using IRQ 38744032 for MSI
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge0: Disabling fastboot
miibus0: <MII bus> numa-domain 0 on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0: OUI 0x001be9, model 0x0022, rev. 0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: bpf attached
bge0: Ethernet address: 2c:09:4d:00:08:fa
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000030000-0x602400003ffff,0x6024000040000-0x602400004ffff,0x6024000050000-0x602400005ffff irq 1032185 at device 0.1 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000030000, len 65536
mmu_radix_mapdev_attr(0x6024000030000, 65536, 0)
ofw_pci mapdev: start 6024000040000, len 65536
mmu_radix_mapdev_attr(0x6024000040000, 65536, 0)
bge1: APE FW version: NCSI v0.6.0.18
bge1: attempting to allocate 1 MSI vectors (8 supported)
bge1: using IRQ 38744033 for MSI
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge1: Disabling fastboot
miibus1: <MII bus> numa-domain 0 on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1: OUI 0x001be9, model 0x0022, rev. 0
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: bpf attached
bge1: Ethernet address: 2c:09:4d:00:08:fb
bge2: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000060000-0x602400006ffff,0x6024000070000-0x602400007ffff,0x6024000080000-0x602400008ffff irq 1032184 at device 0.2 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000060000, len 65536
mmu_radix_mapdev_attr(0x6024000060000, 65536, 0)
ofw_pci mapdev: start 6024000070000, len 65536
mmu_radix_mapdev_attr(0x6024000070000, 65536, 0)
bge2: APE FW version: NCSI v0.6.0.18
bge2: attempting to allocate 1 MSI vectors (8 supported)
bge2: using IRQ 38744034 for MSI
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge2: Disabling fastboot
miibus2: <MII bus> numa-domain 0 on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2: OUI 0x001be9, model 0x0022, rev. 0
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: bpf attached
bge2: Ethernet address: 2c:09:4d:00:08:fc
vgapci0: <VGA-compatible display> mem 0x80000000-0x80ffffff,0x81000000-0x8101ffff irq 1030136 at device 0.0 numa-domain 0 on pci13
ofwbus0: <psi@6030203000000> mem 0x6030203000000-0x6030302ffffff irq 1048560 compat ibm,psi (no driver attached)
ofwbus0: <vas@6019100000000> mem 0x6019100000000-0x6019101ffffff,0x6019000000000-0x60190ffffffff,0x8000000000000-0x80000ffffffff,0x20-0x2f compat ibm,power9-vas (no driver attached)
ofwbus0: <vpd> compat ibm,opal-v3-vpd (no driver attached)
powernv_xscom0: <xscom> mem 0x603fc00000000-0x60403ffffffff on ofwbus0
opal_i2cm0: <centaur-i2cm> on powernv_xscom0
iichb0: <opal-i2c> on opal_i2cm0
iicbus0: <OFW I2C bus> on iichb0
iic0: <I2C generic I/O> on iicbus0
iicbus0: <unknown card> at addr 0xa0
iichb1: <opal-i2c> on opal_i2cm0
iicbus1: <OFW I2C bus> on iichb1
iic1: <I2C generic I/O> on iicbus1
iicbus1: <unknown card> at addr 0xa0
opal_i2cm1: <centaur-i2cm> on powernv_xscom0
iichb2: <opal-i2c> on opal_i2cm1
iicbus2: <OFW I2C bus> on iichb2
iic2: <I2C generic I/O> on iicbus2
iicbus2: <unknown card> at addr 0xa0
opal_i2cm2: <centaur-i2cm> on powernv_xscom0
iichb3: <opal-i2c> on opal_i2cm2
iicbus3: <OFW I2C bus> on iichb3
iic3: <I2C generic I/O> on iicbus3
iicbus3: <unknown card> at addr 0xa0
iichb4: <opal-i2c> on opal_i2cm2
iicbus4: <OFW I2C bus> on iichb4
iic4: <I2C generic I/O> on iicbus4
iicbus4: <unknown card> at addr 0xa8
ossl0: <OpenSSL crypto> on nexus0
crypto: assign ossl0 driver id 0, flags 0xe000000
crypto: assign cryptosoft0 driver id 1, flags 0x6000000
mmu_radix_mapdev_attr(0x60302031c0000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c1000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c2000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c3000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c4000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c5000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c7000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c8000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c9000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031ca000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cb000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cc000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cd000, 4096, 0)
mmu_radix_mapdev_attr(0x600c30ffe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c30fff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c327fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c327ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c347fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c347ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c36ffe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c36fff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c387fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c387ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c3a7fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c3a7ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300010000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300020000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300030000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300040000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300050000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300060000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300070000, 65536, 0)
mmu_radix_mapdev_attr(0x600c340000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c360000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380010000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380020000, 65536, 0)
procfs registered
Timecounter "timebase" frequency 512000000 Hz quality 1000
Event timer "decrementer" frequency 512000000 Hz quality 1000
Timecounters tick every 1.000 msec
vlan: initialized, using hash tables with chaining
lo0: bpf attached
IPsec: Initialized Security Association Processing.
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 262144
pci0: driver added
GEOM: new disk opalflash0
usbus0: 5.0Gbps Super Speed USB v3.0
ahcich0: AHCI reset...
pci1: driver added
pci2: driver added
pci3: driver added
pci4: driver added
found-> vendor=0x1000, dev=0x0097, revid=0x02
        domain=11, bus=1, slot=0, func=0
        class=01-07-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0146, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=248
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit, vector masks
        MSI-X supports 96 messages in map 0x14
pci11:1:0:0: reprobing on driver added
mpr0: <Avago Technologies (LSI) SAS3008> mem 0x80140000-0x8014ffff,0x80100000-0x8013ffff irq 1040376 at device 0.0 numa-domain 0 on pci4
pci4: child mpr0 requested type 3 for rid 0x10, but the BAR says it is an ioport
ofw_pci mapdev: start 600c080140000, len 65536
mmu_radix_mapdev_attr(0x600c080140000, 65536, 0)
mpr0: attempting to allocate 1 MSI-X vectors (96 supported)
mpr0: using IRQ 35637240 for MSI-X
ugen0.1: <(0x104c) XHCI root HUB> at usbus0
uhub0 numa-domain 0 on usbus0
uhub0: <(0x104c) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ahcich0: SATA connect timeout time=10000us status=00000000
ahcich0: AHCI reset: device not found
random: unblocking device.
ahcich1: AHCI reset...
ahcich1: SATA connect time=100us status=00000123
ahcich1: AHCI reset: device found
ahcich2: AHCI reset...
ahcich2: SATA connect timeout time=10000us status=00000000
ahcich2: AHCI reset: device not found
ahcich3: AHCI reset...
ahcich3: SATA connect timeout time=10000us status=00000000
ahcich3: AHCI reset: device not found
ahcich1: AHCI reset: device ready after 100ms
mpr0: Firmware: 16.00.01.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
mpr0: IOC Facts allocation failed with error 12
device_attach: mpr0 attach returned 12
pci5: driver added
pci6: driver added
pci7: driver added
pci8: driver added
pci9: driver added
pci10: driver added
pci11: driver added
pci12: driver added
pci13: driver added
Trying to mount root from ufs:/dev/ada0p2 []...
Adding CPU 0, hwref=4, awake=1
Waking up CPU 1 (dev=5)
Adding CPU 1, hwref=5, awake=1
Waking up CPU 2 (dev=6)
Adding CPU 2, hwref=6, awake=1
Waking up CPU 3 (dev=7)
Adding CPU 3, hwref=7, awake=1
Waking up CPU 4 (dev=24)
Adding CPU 4, hwref=24, awake=1
Waking up CPU 5 (dev=25)
Adding CPU 5, hwref=25, awake=1
Waking up CPU 6 (dev=26)
Adding CPU 6, hwref=26, awake=1
Waking up CPU 7 (dev=27)
Adding CPU 7, hwref=27, awake=1
Waking up CPU 8 (dev=14)
Adding CPU 8, hwref=14, awake=1
Waking up CPU 9 (dev=15)
Adding CPU 9, hwref=15, awake=1
Waking up CPU 10 (dev=16)
Adding CPU 10, hwref=16, awake=1
Waking up CPU 11 (dev=17)
Adding CPU 11, hwref=17, awake=1
Waking up CPU 12 (dev=c)
Adding CPU 12, hwref=c, awake=1
Waking up CPU 13 (dev=d)
Adding CPU 13, hwref=d, awake=1
Waking up CPU 14 (dev=e)
Adding CPU 14, hwref=e, awake=1
Waking up CPU 15 (dev=f)
Adding CPU 15, hwref=f, awake=1
SMP: AP CPU #1 launched
SMP: AP CPU #13 launched
SMP: AP CPU #7 launched
SMP: AP CPU #11 launched
SMP: AP CPU #8 launched
SMP: AP CPU #6 launched
SMP: AP CPU #9 launched
SMP: AP CPU #14 launched
SMP: AP CPU #4 launched
SMP: AP CPU #3 launched
SMP: AP CPU #15 launched
SMP: AP CPU #2 launched
SMP: AP CPU #5 launched
SMP: AP CPU #12 launched
SMP: AP CPU #10 launched
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
ada0: Serial Number JK11A8B9H82U0F
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors)
mmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
pass0 at ahcich1 bus 0 scbus1 target 0 lun 0
pass0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
pass0: Serial Nmmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
umber JK11A8B9H82U0F
pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
pass0: Command Queueing enabled
WARNING: WITNESS option enabled, expect reduced performance.
GEOM: new disk ada0
GEOM: ada0: the secondary GPT header is not in the last LBA.
GEOM: diskid/DISK-JK11A8B9H82U0F: the secondary GPT header is not in the last LBA.
WARNING: / was not properly dismounted
opal0: providing initial system time
failed to load CTF for kernel: 2
start_init: trying /sbin/init
uhub0: 8 ports with 8 removable, self powered
Waiting 30s for the root mount holders: usbus0..............................
Setting hostuuid: ce1c3c60-3bec-48a7-8c41-adc4cb68bbb1.
Setting hostid: 0xfc69d1ea.
Starting file system checks:
** SU+J Recovering /dev/ada0p2
** Reading 655589376 byte journal from inode 4.
** Building recovery table.
** Resolving unreferenced inode list.
** Processing journal entries.
/dev/ada0p2:
**** FILE SYSTEM MARKED CLEAN ****
/dev/ada0p1: 2834 files, 97 MiB free (197923 clusters)
FIXED
/dev/ada0p1: MARKING FILE SYSTEM CLEAN
Mounting local filesystems:.
adrian added a comment.Wed, Feb 18, 3:31 PM

oh interesting. is this different from straight -head? If it is then i need to fix it. If it isn't then it's expected behaviour!

Right now this stack is aiming to be bug for bug whilst I refactor it. the delta to your diffs / arm64/amd64 style bounce code is around ~1000 lines, and tonight I'll work on reducing it to under 1000 lines.

tpearson_raptorengineering.com added a comment.Wed, Feb 18, 3:39 PM

Yeah, here's the boot from a stock kernel:

GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
---<<BOOT>>---
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 16.0-CURRENT #1 main-n282039-a4aa7df767e0: Fri Nov 21 01:22:28 CST 2025
    root@freebsd-ppc64le-dev:/usr/obj/usr/src/powerpc.powerpc64le/sys/GENERIC64LE powerpc
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
Preloaded elf kernel "kernel" at 0xc0000000045e7040.
cpu0: IBM POWER9 revision 2.3, 2700.00 MHz
cpu0: Features dc007182<PPC32,PPC64,ALTIVEC,FPU,MMU,SMT,ISNOOP,ARCH205,ARCH206,VSX,TRUELE>
cpu0: Features2 bee00000<ARCH207,DSCR,ISEL,TAR,VCRYPTO,ARCH300,IEEE128,DARN>
real memory  = 33997053952 (32422 MB)
available KVA = 34359619583 (32767 MB)
Physical memory chunk(s):
0x0000000000003000 - 0x0000000000002fff, 0 bytes (0 pages)
0x000000000000e000 - 0x000000000000ffff, 8192 bytes (2 pages)
0x0000000000094000 - 0x0000000000ffffff, 16171008 bytes (3948 pages)
0x0000000100000000 - 0x00000007a2042fff, 28487987200 bytes (6955075 pages)
0x00000007d0006000 - 0x00000007fc72dfff, 745701376 bytes (182056 pages)
0x00000007fdc00000 - 0x00000007ff79ffff, 28966912 bytes (7072 pages)
0x00000007ff7d2000 - 0x00000007ff7effff, 122880 bytes (30 pages)
avail memory = 29190139904 (27837 MB)
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
cpu0: dev=4 domain=0  (BSP)
cpu1: dev=5 domain=0
cpu2: dev=6 domain=0
cpu3: dev=7 domain=0
cpu4: dev=24 domain=0
cpu5: dev=25 domain=0
cpu6: dev=26 domain=0
cpu7: dev=27 domain=0
cpu8: dev=14 domain=0
cpu9: dev=15 domain=0
cpu10: dev=16 domain=0
cpu11: dev=17 domain=0
cpu12: dev=c domain=0
cpu13: dev=d domain=0
cpu14: dev=e domain=0
cpu15: dev=f domain=0
Installed OPAL HMI handler.
random: registering fast source PowerISA DARN random number generator
random: fast provider: "PowerISA DARN random number generator"
random: no preloaded entropy cache
random: no platform bootloader entropy
arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled.
VIMAGE (virtualized network stack) enabled
hostuuid: using 00000000-0000-0000-0000-000000000000
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
ULE: setup cpu 7
ULE: setup cpu 8
ULE: setup cpu 9
ULE: setup cpu 10
ULE: setup cpu 11
ULE: setup cpu 12
ULE: setup cpu 13
ULE: setup cpu 14
ULE: setup cpu 15
firmware: 't4fw_cfg' version 0: 4523 bytes loaded at 0xc000000003cda268
firmware: 't4fw_cfg_uwire' version 0: 21433 bytes loaded at 0xc000000003cdb413
firmware: 't4fw' version 0: 570880 bytes loaded at 0xc000000003ce07cc
firmware: 't5fw_cfg' version 0: 5902 bytes loaded at 0xc000000003d6bdcc
firmware: 't5fw_cfg_uwire' version 0: 21859 bytes loaded at 0xc000000003d6d4da
firmware: 't5fw' version 0: 678912 bytes loaded at 0xc000000003d72a3d
firmware: 't6fw_cfg' version 0: 5960 bytes loaded at 0xc000000003e1863d
firmware: 't6fw_cfg_uwire' version 0: 22693 bytes loaded at 0xc000000003e19d85
firmware: 't6fw' version 0: 730624 bytes loaded at 0xc000000003e1f62a
firmware: 't7fw_cfg' version 0: 24043 bytes loaded at 0xc000000003ed1c2a
firmware: 't7fw_cfg_uwire' version 0: 24019 bytes loaded at 0xc000000003ed7a15
random: entropy device external interface
wlan: <802.11 Link Layer>
firmware: 'ice_ddp' version 0: 1352580 bytes loaded at 0xc000000003edd7e8
firmware: 'isp_2400' version 1: 204396 bytes loaded at 0xc0000000039a7df8
firmware: 'isp_2500' version 1: 205736 bytes loaded at 0xc0000000039e8a4c
firmware: 'isp_2600' version 1: 290876 bytes loaded at 0xc000000003a27518
firmware: 'isp_2700' version 1: 343552 bytes loaded at 0xc000000003a7e9a0
firmware: 'isp_2800' version 1: 380416 bytes loaded at 0xc000000003ae99a0
[ath_hal] loaded
kbd0 at kbdmux0
mem: <memory>
null: <full device, null device, zero device>
openfirm: <Open Firmware control device>
sound_global_init: snd_unit=-1 snd_vchans_enable=1 latency=2 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25
tcp_log: tcp_log device
crypto: <crypto core>
ofwbus0: <Open Firmware Device Tree> on nexus0
opal0: <OPAL Abstraction Firmware> irq 1048560,1048561,1048562,1048563,1048564,1048565,1048567,1048568,1048569,1048570,1048571,1048572,1048573,1044478,1044479,1040382,1040383,1038334,1038335,1036286,1036287,1032190,1032191,1030142,1030143 on ofwbus0
opal0: registered as a time-of-day clock, resolution 0.002000s
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
opal0: no domain found
ofwbus0: no default resources for rid = 25, type = 1
simplebus0: <Flattened device tree simple bus> mem 0x6030000000000-0x60300ffffffff on ofwbus0
pcib0: <OPAL Host-PCI bridge> mem 0x600c3c0000000-0x600c3c0000fff,0x600c300000000-0x600c30fffffff on ofwbus0
pcib0: OPAL ID 0
mmu_radix_mapdev_attr(0x600c3c0000000, 4096, 0)
pcib0: Mapping 0-0x800000000 for DMA
pcib0: Supports 4088 MSIs starting at 1040384
pci0: <OFW PCI bus> numa-domain 0 on pcib0
pci0: domain=0, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=0, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=34
pcib1: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci0
pcib1:   domain            0
pcib1:   secondary bus     1
pcib1:   subordinate bus   34
pcib1:   memory decode     0x80000000-0xffefffff
pcib1:   prefetched decode 0x6000000000000-0x6003fbfffffff
pci1: <OFW PCI bus> numa-domain 0 on pcib1
pcib1: allocated bus range (1-1) for rid 0 of pci1
pci1: domain=0, physical bus=1
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=1, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=2, subbus=34
pcib1: allocated bus range (2-34) for rid 0 of pci0:1:0:0
pcib2: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci1
pcib1: allocated memory range (0x80000000-0xffefffff) for rid 20 of pcib2
pcib1: allocated prefetch range (0x6000000000000-0x6003fbfffffff) for rid 24 of pcib2
pcib2:   domain            0
pcib2:   secondary bus     2
pcib2:   subordinate bus   34
pcib2:   memory decode     0x80000000-0xffefffff
pcib2:   prefetched decode 0x6000000000000-0x6003fbfffffff
pci2: <OFW PCI bus> numa-domain 0 on pcib2
pcib2: allocated bus range (2-2) for rid 0 of pci2
pci2: domain=0, physical bus=2
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=3, subbus=3
pcib2: allocated bus range (3-3) for rid 0 of pci0:2:0:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=1, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=4, subbus=4
pcib2: allocated bus range (4-4) for rid 0 of pci0:2:1:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=2, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=5, subbus=9
pcib2: allocated bus range (5-9) for rid 0 of pci0:2:2:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=3, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=10, subbus=14
pcib2: allocated bus range (10-14) for rid 0 of pci0:2:3:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=4, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=15, subbus=19
pcib2: allocated bus range (15-19) for rid 0 of pci0:2:4:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=5, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=20, subbus=24
pcib2: allocated bus range (20-24) for rid 0 of pci0:2:5:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=6, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=25, subbus=29
pcib2: allocated bus range (25-29) for rid 0 of pci0:2:6:0
found-> vendor=0x11f8, dev=0x8533, revid=0x00
        domain=0, bus=2, slot=7, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0545, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        secbus=30, subbus=34
pcib2: allocated bus range (30-34) for rid 0 of pci0:2:7:0
pcib3: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80000000-0x803fffff) for rid 20 of pcib3
pcib2: allocated prefetch range (0x6000000000000-0x600001fffffff) for rid 24 of pcib3
pcib3: attempting to allocate 1 MSI vectors (8 supported)
pcib3: using IRQ 34594816 for MSI
pcib3: HotPlug command: 03c0 -> 17eb
pcib3:   domain            0
pcib3:   secondary bus     3
pcib3:   subordinate bus   3
pcib3:   memory decode     0x80000000-0x803fffff
pcib3:   prefetched decode 0x6000000000000-0x600001fffffff
pcib4: <OFW PCI-PCI bridge> at device 1.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80400000-0x807fffff) for rid 20 of pcib4
pcib2: allocated prefetch range (0x6000020000000-0x600003fffffff) for rid 24 of pcib4
pcib4: attempting to allocate 1 MSI vectors (8 supported)
pcib4: using IRQ 34594817 for MSI
pcib4: HotPlug command: 03c0 -> 17eb
pcib4:   domain            0
pcib4:   secondary bus     4
pcib4:   subordinate bus   4
pcib4:   memory decode     0x80400000-0x807fffff
pcib4:   prefetched decode 0x6000020000000-0x600003fffffff
pcib5: <OFW PCI-PCI bridge> at device 2.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80800000-0x80bfffff) for rid 20 of pcib5
pcib2: allocated prefetch range (0x6000040000000-0x600005fffffff) for rid 24 of pcib5
pcib5: attempting to allocate 1 MSI vectors (8 supported)
pcib5: using IRQ 34594818 for MSI
pcib5: HotPlug command: 03c0 -> 17eb
pcib5:   domain            0
pcib5:   secondary bus     5
pcib5:   subordinate bus   9
pcib5:   memory decode     0x80800000-0x80bfffff
pcib5:   prefetched decode 0x6000040000000-0x600005fffffff
pcib6: <OFW PCI-PCI bridge> at device 3.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x80c00000-0x80ffffff) for rid 20 of pcib6
pcib2: allocated prefetch range (0x6000060000000-0x600007fffffff) for rid 24 of pcib6
pcib6: attempting to allocate 1 MSI vectors (8 supported)
pcib6: using IRQ 34594819 for MSI
pcib6: HotPlug command: 03c0 -> 17eb
pcib6:   domain            0
pcib6:   secondary bus     10
pcib6:   subordinate bus   14
pcib6:   memory decode     0x80c00000-0x80ffffff
pcib6:   prefetched decode 0x6000060000000-0x600007fffffff
pcib7: <OFW PCI-PCI bridge> at device 4.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81000000-0x813fffff) for rid 20 of pcib7
pcib2: allocated prefetch range (0x6000080000000-0x600009fffffff) for rid 24 of pcib7
pcib7: attempting to allocate 1 MSI vectors (8 supported)
pcib7: using IRQ 34594820 for MSI
pcib7: HotPlug command: 03c0 -> 17eb
pcib7:   domain            0
pcib7:   secondary bus     15
pcib7:   subordinate bus   19
pcib7:   memory decode     0x81000000-0x813fffff
pcib7:   prefetched decode 0x6000080000000-0x600009fffffff
pcib8: <OFW PCI-PCI bridge> at device 5.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81400000-0x817fffff) for rid 20 of pcib8
pcib2: allocated prefetch range (0x60000a0000000-0x60000bfffffff) for rid 24 of pcib8
pcib8: attempting to allocate 1 MSI vectors (8 supported)
pcib8: using IRQ 34594821 for MSI
pcib8: HotPlug command: 03c0 -> 17eb
pcib8:   domain            0
pcib8:   secondary bus     20
pcib8:   subordinate bus   24
pcib8:   memory decode     0x81400000-0x817fffff
pcib8:   prefetched decode 0x60000a0000000-0x60000bfffffff
pcib9: <OFW PCI-PCI bridge> at device 6.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81800000-0x81bfffff) for rid 20 of pcib9
pcib2: allocated prefetch range (0x60000c0000000-0x60000dfffffff) for rid 24 of pcib9
pcib9: attempting to allocate 1 MSI vectors (8 supported)
pcib9: using IRQ 34594822 for MSI
pcib9: HotPlug command: 03c0 -> 17eb
pcib9:   domain            0
pcib9:   secondary bus     25
pcib9:   subordinate bus   29
pcib9:   memory decode     0x81800000-0x81bfffff
pcib9:   prefetched decode 0x60000c0000000-0x60000dfffffff
pcib10: <OFW PCI-PCI bridge> at device 7.0 numa-domain 0 on pci2
pcib2: allocated memory range (0x81c00000-0x81ffffff) for rid 20 of pcib10
pcib2: allocated prefetch range (0x60000e0000000-0x60000ffffffff) for rid 24 of pcib10
pcib10: attempting to allocate 1 MSI vectors (8 supported)
pcib10: using IRQ 34594823 for MSI
pcib10: HotPlug command: 03c0 -> 17eb
pcib10:   domain            0
pcib10:   secondary bus     30
pcib10:   subordinate bus   34
pcib10:   memory decode     0x81c00000-0x81ffffff
pcib10:   prefetched decode 0x60000e0000000-0x60000ffffffff
pcib11: <OPAL Host-PCI bridge> mem 0x600c3c0100000-0x600c3c0100fff,0x600c320000000-0x600c32fffffff on ofwbus0
pcib11: OPAL ID 0x1
mmu_radix_mapdev_attr(0x600c3c0100000, 4096, 0)
pcib11: Mapping 0-0x800000000 for DMA
pcib11: Supports 2040 MSIs starting at 1038336
pci3: <OFW PCI bus> numa-domain 0 on pcib11
pci3: domain=11, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=11, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib12: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci3
pcib12:   domain            11
pcib12:   secondary bus     1
pcib12:   subordinate bus   1
pcib12:   memory decode     0x80000000-0xffefffff
pci4: <OFW PCI bus> numa-domain 0 on pcib12
pcib12: allocated bus range (1-1) for rid 0 of pci4
pci4: domain=11, physical bus=1
found-> vendor=0x1000, dev=0x0097, revid=0x02
        domain=11, bus=1, slot=0, func=0
        class=01-07-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit, vector masks
        MSI-X supports 96 messages in map 0x14
        map[10]: type I/O Port, range 32, base 0, size  8, port disabled
        map[14]: type Memory, range 64, base 0x80140000, size 16, enabled
pcib12: allocated memory range (0x80140000-0x8014ffff) for rid 14 of pci11:1:0:0
        map[1c]: type Memory, range 64, base 0x80100000, size 18, enabled
pcib12: allocated memory range (0x80100000-0x8013ffff) for rid 1c of pci11:1:0:0
pcib13: <OPAL Host-PCI bridge> mem 0x600c3c0200000-0x600c3c0200fff,0x600c340000000-0x600c34fffffff on ofwbus0
pcib13: OPAL ID 0x2
mmu_radix_mapdev_attr(0x600c3c0200000, 4096, 0)
pcib13: Mapping 0-0x800000000 for DMA
pcib13: Supports 2040 MSIs starting at 1036288
pci5: <OFW PCI bus> numa-domain 0 on pcib13
pci5: domain=13, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=13, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib14: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci5
pcib14:   domain            13
pcib14:   secondary bus     1
pcib14:   subordinate bus   1
pcib14:   memory decode     0x80000000-0xffefffff
pci6: <OFW PCI bus> numa-domain 0 on pcib14
pcib14: allocated bus range (1-1) for rid 0 of pci6
pci6: domain=13, physical bus=1
found-> vendor=0x1b4b, dev=0x9235, revid=0x11
        domain=13, bus=1, slot=0, func=0
        class=01-06-01, hdrtype=0x00, mfdev=0
        cmdreg=0x0543, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 1 message
        map[10]: type I/O Port, range 32, base 0x8000, size  3, enabled
pcib14: failed to allocate initial I/O port window (0x8000-0x8007,0x8)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x8)
pci6: pci13:1:0:0 bar 0x10 failed to allocate
        map[14]: type I/O Port, range 32, base 0x8040, size  2, port disabled
pcib14: failed to allocate initial I/O port window (0x8040-0x8043,0x4)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x4)
pci6: pci13:1:0:0 bar 0x14 failed to allocate
        map[18]: type I/O Port, range 32, base 0x8100, size  3, port disabled
pcib14: failed to allocate initial I/O port window (0x8100-0x8107,0x8)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x8)
pci6: pci13:1:0:0 bar 0x18 failed to allocate
        map[1c]: type I/O Port, range 32, base 0x8140, size  2, port disabled
pcib14: failed to allocate initial I/O port window (0x8140-0x8143,0x4)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x4)
pci6: pci13:1:0:0 bar 0x1c failed to allocate
        map[20]: type I/O Port, range 32, base 0x800000, size  5, port disabled
pcib14: failed to allocate initial I/O port window (0x800000-0x80001f,0x20)
pcib14: failed to allocate initial I/O port window (0-0xffffffff,0x20)
pci6: pci13:1:0:0 bar 0x20 failed to allocate
        map[24]: type Memory, range 32, base 0x80000000, size 11, enabled
pcib14: allocated memory range (0x80000000-0x800007ff) for rid 24 of pci13:1:0:0
pcib15: <OPAL Host-PCI bridge> mem 0x600c3c0300000-0x600c3c0300fff,0x600c360000000-0x600c36fffffff on ofwbus0
pcib15: OPAL ID 0x3
mmu_radix_mapdev_attr(0x600c3c0300000, 4096, 0)
pcib15: Mapping 0-0x800000000 for DMA
pcib15: Supports 4088 MSIs starting at 1032192
pci7: <OFW PCI bus> numa-domain 0 on pcib15
pci7: domain=15, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=15, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib16: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci7
pcib16:   domain            15
pcib16:   secondary bus     1
pcib16:   subordinate bus   1
pcib16:   memory decode     0x80000000-0xffefffff
pci8: <OFW PCI bus> numa-domain 0 on pcib16
pcib16: allocated bus range (1-1) for rid 0 of pci8
pci8: domain=15, physical bus=1
found-> vendor=0x104c, dev=0x8241, revid=0x02
        domain=15, bus=1, slot=0, func=0
        class=0c-03-30, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 8 messages in map 0x18
        map[10]: type Memory, range 64, base 0x80000000, size 16, enabled
pcib16: allocated memory range (0x80000000-0x8000ffff) for rid 10 of pci15:1:0:0
        map[18]: type Memory, range 64, base 0x80010000, size 13, enabled
pcib16: allocated memory range (0x80010000-0x80011fff) for rid 18 of pci15:1:0:0
pcib17: <OPAL Host-PCI bridge> mem 0x600c3c0400000-0x600c3c0400fff,0x600c380000000-0x600c38fffffff on ofwbus0
pcib17: OPAL ID 0x4
mmu_radix_mapdev_attr(0x600c3c0400000, 4096, 0)
pcib17: Mapping 0-0x800000000 for DMA
pcib17: Supports 2040 MSIs starting at 1030144
pci9: <OFW PCI bus> numa-domain 0 on pcib17
pci9: domain=17, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=17, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=1
pcib18: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci9
pcib18:   domain            17
pcib18:   secondary bus     1
pcib18:   subordinate bus   1
pcib18:   memory decode     0x80000000-0xffefffff
pcib18:   prefetched decode 0x6024000000000-0x6027f7fffffff
pci10: <OFW PCI bus> numa-domain 0 on pcib18
pcib18: allocated bus range (1-1) for rid 0 of pci10
pci10: domain=17, physical bus=1
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=0
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000000000, size 16, enabled
pcib18: allocated prefetch range (0x6024000000000-0x602400000ffff) for rid 10 of pci17:1:0:0
        map[18]: type Prefetchable Memory, range 64, base 0x6024000010000, size 16, enabled
pcib18: allocated prefetch range (0x6024000010000-0x602400001ffff) for rid 18 of pci17:1:0:0
        map[20]: type Prefetchable Memory, range 64, base 0x6024000020000, size 16, enabled
pcib18: allocated prefetch range (0x6024000020000-0x602400002ffff) for rid 20 of pci17:1:0:0
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=1
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=b, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000030000, size 16, enabled
pcib18: allocated prefetch range (0x6024000030000-0x602400003ffff) for rid 10 of pci17:1:0:1
        map[18]: type Prefetchable Memory, range 64, base 0x6024000040000, size 16, enabled
pcib18: allocated prefetch range (0x6024000040000-0x602400004ffff) for rid 18 of pci17:1:0:1
        map[20]: type Prefetchable Memory, range 64, base 0x6024000050000, size 16, enabled
pcib18: allocated prefetch range (0x6024000050000-0x602400005ffff) for rid 20 of pci17:1:0:1
found-> vendor=0x14e4, dev=0x1657, revid=0x00
        domain=17, bus=1, slot=0, func=2
        class=02-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0142, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D3  current D0
        MSI supports 8 messages, 64 bit
        MSI-X supports 17 messages in map 0x20
        map[10]: type Prefetchable Memory, range 64, base 0x6024000060000, size 16, enabled
pcib18: allocated prefetch range (0x6024000060000-0x602400006ffff) for rid 10 of pci17:1:0:2
        map[18]: type Prefetchable Memory, range 64, base 0x6024000070000, size 16, enabled
pcib18: allocated prefetch range (0x6024000070000-0x602400007ffff) for rid 18 of pci17:1:0:2
        map[20]: type Prefetchable Memory, range 64, base 0x6024000080000, size 16, enabled
pcib18: allocated prefetch range (0x6024000080000-0x602400008ffff) for rid 20 of pci17:1:0:2
pcib19: <OPAL Host-PCI bridge> mem 0x600c3c0500000-0x600c3c0500fff,0x600c3a0000000-0x600c3afffffff on ofwbus0
pcib19: OPAL ID 0x5
mmu_radix_mapdev_attr(0x600c3c0500000, 4096, 0)
pcib19: Mapping 0-0x800000000 for DMA
pcib19: Supports 2040 MSIs starting at 1028096
pci11: <OFW PCI bus> numa-domain 0 on pcib19
pci11: domain=19, physical bus=0
found-> vendor=0x1014, dev=0x04c1, revid=0x00
        domain=19, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0103, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 3  supports D0 D3  current D0
        secbus=1, subbus=2
pcib20: <OFW PCI-PCI bridge> at device 0.0 numa-domain 0 on pci11
pcib20:   domain            19
pcib20:   secondary bus     1
pcib20:   subordinate bus   2
pcib20:   memory decode     0x80000000-0xffefffff
pci12: <OFW PCI bus> numa-domain 0 on pcib20
pcib20: allocated bus range (1-1) for rid 0 of pci12
pci12: domain=19, physical bus=1
found-> vendor=0x1a03, dev=0x1150, revid=0x04
        domain=19, bus=1, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0143, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit
        secbus=2, subbus=2
pcib20: allocated bus range (2-2) for rid 0 of pci19:1:0:0
pcib21: <OFW PCI-PCI bridge> irq 1030136 at device 0.0 numa-domain 0 on pci12
pcib20: allocated memory range (0x80000000-0xffefffff) for rid 20 of pcib21
pcib21:   domain            19
pcib21:   secondary bus     2
pcib21:   subordinate bus   2
pcib21:   memory decode     0x80000000-0xffefffff
pci13: <OFW PCI bus> numa-domain 0 on pcib21
pcib21: allocated bus range (2-2) for rid 0 of pci13
pci13: domain=19, physical bus=2
found-> vendor=0x1a03, dev=0x2000, revid=0x41
        domain=19, bus=2, slot=0, func=0
        class=03-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0142, statreg=0x0210, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=0
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 2 messages, 64 bit
        map[10]: type Memory, range 32, base 0x80000000, size 24, enabled
pcib21: allocated memory range (0x80000000-0x80ffffff) for rid 10 of pci19:2:0:0
        map[14]: type Memory, range 32, base 0x81000000, size 17, enabled
pcib21: allocated memory range (0x81000000-0x8101ffff) for rid 14 of pci19:2:0:0
        map[18]: type I/O Port, range 32, base 0, size  7, port disabled
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
cpu0: Nominal frequency 2700Mhz
cpu1: <Open Firmware CPU> on cpulist0
cpu1: Nominal frequency 2700Mhz
cpu2: <Open Firmware CPU> on cpulist0
cpu2: Nominal frequency 2700Mhz
cpu3: <Open Firmware CPU> on cpulist0
cpu3: Nominal frequency 2700Mhz
xive0: <External Interrupt Virtualization Engine> mem 0x6030203180000-0x603020318ffff,0x6030203190000-0x603020319ffff,0x60302031a0000-0x60302031affff,0x60302031b0000-0x60302031bffff on ofwbus0
mmu_radix_mapdev_attr(0x6030203190000, 65536, 0)
mmu_radix_mapdev_attr(0x6010000410000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000400000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000430000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000420000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000450000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000440000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000470000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000460000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000490000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000480000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004b0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004a0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004d0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004c0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004f0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100004e0000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000510000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000500000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000530000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000520000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000550000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000540000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000570000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000560000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000590000, 131072, 0)
mmu_radix_mapdev_attr(0x6010000580000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005b0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005a0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005d0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005c0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005f0000, 131072, 0)
mmu_radix_mapdev_attr(0x60100005e0000, 131072, 0)
xivevc0: <External Interrupt Virtualization Engine Root> on ofwbus0
pmcr0: <Power Management Control Register> on cpu0
cpufreq0: <CPU frequency control> on cpu0
pmcr1: <Power Management Control Register> on cpu1
cpufreq1: <CPU frequency control> on cpu1
pmcr2: <Power Management Control Register> on cpu2
cpufreq2: <CPU frequency control> on cpu2
pmcr3: <Power Management Control Register> on cpu3
cpufreq3: <CPU frequency control> on cpu3
ofwbus0: <bmc> compat ibm,ast2500,openbmc (no driver attached)
opalcons0: <OPAL Consoles> on opal0
uart0: <OPAL Serial Port> on opalcons0
uart0: console
opalflash0: <OPAL System Flash> on opal0
opalsens0: <OPAL Sensors> on opal0
opal_sensor0: <OPAL sensor> on opalsens0
opal_sensor1: <OPAL sensor> on opalsens0
opal_sensor2: <OPAL sensor> on opalsens0
opal_sensor3: <OPAL sensor> on opalsens0
opal_sensor4: <OPAL sensor> on opalsens0
opal_sensor5: <OPAL sensor> on opalsens0
opal_sensor6: <OPAL sensor> on opalsens0
opal_sensor7: <OPAL sensor> on opalsens0
opal_sensor8: <OPAL sensor> on opalsens0
opal_sensor9: <OPAL sensor> on opalsens0
opal_sensor10: <OPAL sensor> on opalsens0
opal_sensor11: <OPAL sensor> on opalsens0
opal_sensor12: <OPAL sensor> on opalsens0
opal_sensor13: <OPAL sensor> on opalsens0
opal_sensor14: <OPAL sensor> on opalsens0
opal_sensor15: <OPAL sensor> on opalsens0
opal_sensor16: <OPAL sensor> on opalsens0
opal_sensor17: <OPAL sensor> on opalsens0
opal_sensor18: <OPAL sensor> on opalsens0
opal_sensor19: <OPAL sensor> on opalsens0
opal_sensor20: <OPAL sensor> on opalsens0
opal_sensor21: <OPAL sensor> on opalsens0
opal_sensor22: <OPAL sensor> on opalsens0
opal_sensor23: <OPAL sensor> on opalsens0
opal_sensor24: <OPAL sensor> on opalsens0
opal_sensor25: <OPAL sensor> on opalsens0
opal_sensor26: <OPAL sensor> on opalsens0
opal_sensor27: <OPAL sensor> on opalsens0
opal_sensor28: <OPAL sensor> on opalsens0
opal_sensor29: <OPAL sensor> on opalsens0
opal_sensor30: <OPAL sensor> on opalsens0
opal_sensor31: <OPAL sensor> on opalsens0
opal_sensor32: <OPAL sensor> on opalsens0
opal_sensor33: <OPAL sensor> on opalsens0
ofwbus0: <ibm,secureboot> compat ibm,secureboot (no driver attached)
ofwbus0: <imc-counters> compat ibm,opal-in-memory-counters (no driver attached)
simplebus0: <lpc-controller@c0012000> mem 0xc0012000-0xc00120ff compat ibm,power9-lpc-controller (no driver attached)
simplebus0: <lpc@0> compat ibm,power9-lpc (no driver attached)
simplebus0: <opb-arbiter@c0011000> mem 0xc0011000-0xc0011007 compat ibm,power9-lpcm-opb-arbiter (no driver attached)
simplebus0: <opb-master@c0010000> mem 0xc0010000-0xc001005f compat ibm,power9-lpcm-opb-master (no driver attached)
mpr0: <Avago Technologies (LSI) SAS3008> mem 0x80140000-0x8014ffff,0x80100000-0x8013ffff irq 1040376 at device 0.0 numa-domain 0 on pci4
pci4: child mpr0 requested type 3 for rid 0x10, but the BAR says it is an ioport
ofw_pci mapdev: start 600c080140000, len 65536
mmu_radix_mapdev_attr(0x600c080140000, 65536, 0)
mpr0: attempting to allocate 1 MSI-X vectors (96 supported)
mpr0: using IRQ 35637240 for MSI-X
mpr0: Firmware: 16.00.01.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
mpr0: IOC Facts allocation failed with error 12
device_attach: mpr0 attach returned 12
ahci0: <Marvell 88SE9235 AHCI SATA controller> mem 0x80000000-0x800007ff irq 1038328 at device 0.0 numa-domain 0 on pci6
ofw_pci mapdev: start 600c100000000, len 2048
mmu_radix_mapdev_attr(0x600c100000000, 4096, 0)
ahci0: attempting to allocate 1 MSI vectors (1 supported)
ahci0: using IRQ 36675568 for MSI
ahci0: AHCI v1.00 with 4 6Gbps ports, Port Multiplier supported with FBS
ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 4ports
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: Caps: CPD ESP FBSCP
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich0: FAILURE - create data_map
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: Caps: CPD ESP FBSCP
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich1: FAILURE - create data_map
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich2: Caps: CPD ESP FBSCP
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich2: FAILURE - create data_map
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich3: Caps: CPD ESP FBSCP
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
ahcich3: FAILURE - create data_map
xhci0: <XHCI (generic) USB 3.0 controller> mem 0x80000000-0x8000ffff,0x80010000-0x80011fff irq 1036280 at device 0.0 numa-domain 0 on pci8
ofw_pci mapdev: start 600c180000000, len 65536
mmu_radix_mapdev_attr(0x600c180000000, 65536, 0)
xhci0: 64 bytes context size, 64-bit DMA
xhci0: xECP capabilities <LEGACY,PROTO,PROTO>
ofw_pci mapdev: start 600c180010000, len 8192
mmu_radix_mapdev_attr(0x600c180010000, 8192, 0)
xhci0: attempting to allocate 1 MSI-X vectors (8 supported)
xhci0: using IRQ 37709800 for MSI-X
xhci0: MSI-X enabled
xhci0: (New XHCI DeviceId=0x8241104c)
usbus0 numa-domain 0 on xhci0
xhci0: usbpf: Attached
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000000000-0x602400000ffff,0x6024000010000-0x602400001ffff,0x6024000020000-0x602400002ffff irq 1032184 at device 0.0 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000000000, len 65536
mmu_radix_mapdev_attr(0x6024000000000, 65536, 0)
ofw_pci mapdev: start 6024000010000, len 65536
mmu_radix_mapdev_attr(0x6024000010000, 65536, 0)
bge0: APE FW version: NCSI v0.6.0.18
bge0: attempting to allocate 1 MSI vectors (8 supported)
bge0: using IRQ 38744032 for MSI
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge0: Disabling fastboot
miibus0: <MII bus> numa-domain 0 on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0: OUI 0x001be9, model 0x0022, rev. 0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: bpf attached
bge0: Ethernet address: 2c:09:4d:00:08:fa
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000030000-0x602400003ffff,0x6024000040000-0x602400004ffff,0x6024000050000-0x602400005ffff irq 1032185 at device 0.1 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000030000, len 65536
mmu_radix_mapdev_attr(0x6024000030000, 65536, 0)
ofw_pci mapdev: start 6024000040000, len 65536
mmu_radix_mapdev_attr(0x6024000040000, 65536, 0)
bge1: APE FW version: NCSI v0.6.0.18
bge1: attempting to allocate 1 MSI vectors (8 supported)
bge1: using IRQ 38744033 for MSI
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge1: Disabling fastboot
miibus1: <MII bus> numa-domain 0 on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1: OUI 0x001be9, model 0x0022, rev. 0
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: bpf attached
bge1: Ethernet address: 2c:09:4d:00:08:fb
bge2: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x5719001> mem 0x6024000060000-0x602400006ffff,0x6024000070000-0x602400007ffff,0x6024000080000-0x602400008ffff irq 1032184 at device 0.2 numa-domain 0 on pci10
ofw_pci mapdev: start 6024000060000, len 65536
mmu_radix_mapdev_attr(0x6024000060000, 65536, 0)
ofw_pci mapdev: start 6024000070000, len 65536
mmu_radix_mapdev_attr(0x6024000070000, 65536, 0)
bge2: APE FW version: NCSI v0.6.0.18
bge2: attempting to allocate 1 MSI vectors (8 supported)
bge2: using IRQ 38744034 for MSI
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
bge2: Disabling fastboot
miibus2: <MII bus> numa-domain 0 on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2: OUI 0x001be9, model 0x0022, rev. 0
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: bpf attached
bge2: Ethernet address: 2c:09:4d:00:08:fc
vgapci0: <VGA-compatible display> mem 0x80000000-0x80ffffff,0x81000000-0x8101ffff irq 1030136 at device 0.0 numa-domain 0 on pci13
ofwbus0: <psi@6030203000000> mem 0x6030203000000-0x6030302ffffff irq 1048560 compat ibm,psi (no driver attached)
ofwbus0: <vas@6019100000000> mem 0x6019100000000-0x6019101ffffff,0x6019000000000-0x60190ffffffff,0x8000000000000-0x80000ffffffff,0x20-0x2f compat ibm,power9-vas (no driver attached)
ofwbus0: <vpd> compat ibm,opal-v3-vpd (no driver attached)
powernv_xscom0: <xscom> mem 0x603fc00000000-0x60403ffffffff on ofwbus0
opal_i2cm0: <centaur-i2cm> on powernv_xscom0
iichb0: <opal-i2c> on opal_i2cm0
iicbus0: <OFW I2C bus> on iichb0
iic0: <I2C generic I/O> on iicbus0
iicbus0: <unknown card> at addr 0xa0
iichb1: <opal-i2c> on opal_i2cm0
iicbus1: <OFW I2C bus> on iichb1
iic1: <I2C generic I/O> on iicbus1
iicbus1: <unknown card> at addr 0xa0
opal_i2cm1: <centaur-i2cm> on powernv_xscom0
iichb2: <opal-i2c> on opal_i2cm1
iicbus2: <OFW I2C bus> on iichb2
iic2: <I2C generic I/O> on iicbus2
iicbus2: <unknown card> at addr 0xa0
opal_i2cm2: <centaur-i2cm> on powernv_xscom0
iichb3: <opal-i2c> on opal_i2cm2
iicbus3: <OFW I2C bus> on iichb3
iic3: <I2C generic I/O> on iicbus3
iicbus3: <unknown card> at addr 0xa0
iichb4: <opal-i2c> on opal_i2cm2
iicbus4: <OFW I2C bus> on iichb4
iic4: <I2C generic I/O> on iicbus4
iicbus4: <unknown card> at addr 0xa8
ossl0: <OpenSSL crypto> on nexus0
crypto: assign ossl0 driver id 0, flags 0xe000000
crypto: assign cryptosoft0 driver id 1, flags 0x6000000
mmu_radix_mapdev_attr(0x60302031c0000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c1000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c2000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c3000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c4000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c5000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c7000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c8000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031c9000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031ca000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cb000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cc000, 4096, 0)
mmu_radix_mapdev_attr(0x60302031cd000, 4096, 0)
mmu_radix_mapdev_attr(0x600c30ffe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c30fff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c327fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c327ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c347fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c347ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c36ffe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c36fff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c387fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c387ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c3a7fe0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c3a7ff0000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300010000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300020000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300030000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300040000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300050000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300060000, 65536, 0)
mmu_radix_mapdev_attr(0x600c300070000, 65536, 0)
mmu_radix_mapdev_attr(0x600c340000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c360000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380000000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380010000, 65536, 0)
mmu_radix_mapdev_attr(0x600c380020000, 65536, 0)
procfs registered
Timecounter "timebase" frequency 512000000 Hz quality 1000
Event timer "decrementer" frequency 512000000 Hz quality 1000
Timecounters tick every 1.000 msec
lo0: bpf attached
vlan: initialized, using hash tables with chaining
IPsec: Initialized Security Association Processing.
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 262144
pci0: driver added
GEOM: new disk opalflash0
usbus0: 5.0Gbps Super Speed USB v3.0
ahcich0: AHCI reset...
pci1: driver added
pci2: driver added
pci3: driver added
pci4: driver added
found-> vendor=0x1000, dev=0x0097, revid=0x02
        domain=11, bus=1, slot=0, func=0
        class=01-07-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0146, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=248
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 1 message, 64 bit, vector masks
        MSI-X supports 96 messages in map 0x14
pci11:1:0:0: reprobing on driver added
mpr0: <Avago Technologies (LSI) SAS3008> mem 0x80140000-0x8014ffff,0x80100000-0x8013ffff irq 1040376 at device 0.0 numa-domain 0 on pci4
pci4: child mpr0 requested type 3 for rid 0x10, but the BAR says it is an ioport
ofw_pci mapdev: start 600c080140000, len 65536
mmu_radix_mapdev_attr(0x600c080140000, 65536, 0)
mpr0: attempting to allocate 1 MSI-X vectors (96 supported)
mpr0: using IRQ 35637240 for MSI-X
ugen0.1: <(0x104c) XHCI root HUB> at usbus0
uhub0 numa-domain 0 on usbus0
uhub0: <(0x104c) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ahcich0: SATA connect timeout time=10000us status=00000000
ahcich0: AHCI reset: device not found
ahcich1: AHCI reset...
ahcich1: SATA connect time=100us status=00000123
ahcich1: AHCI reset: device found
ahcich2: AHCI reset...
ahcich2: SATA connect timeout time=10000us status=00000000
ahcich2: AHCI reset: device not found
ahcich3: AHCI reset...
random: unblocking device.
ahcich3: SATA connect timeout time=10000us status=00000000
ahcich3: AHCI reset: device not found
ahcich1: AHCI reset: device ready after 100ms
mpr0: Firmware: 16.00.01.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
mpr0: IOC Facts allocation failed with error 12
device_attach: mpr0 attach returned 12
pci5: driver added
pci6: driver added
pci7: driver added
pci8: driver added
pci9: driver added
pci10: driver added
pci11: driver added
pci12: driver added
pci13: driver added
Trying to mount root from ufs:/dev/ada0p2 []...
Adding CPU 0, hwref=4, awake=1
Waking up CPU 1 (dev=5)
Adding CPU 1, hwref=5, awake=1
Waking up CPU 2 (dev=6)
Adding CPU 2, hwref=6, awake=1
Waking up CPU 3 (dev=7)
Adding CPU 3, hwref=7, awake=1
Waking up CPU 4 (dev=24)
Adding CPU 4, hwref=24, awake=1
Waking up CPU 5 (dev=25)
Adding CPU 5, hwref=25, awake=1
Waking up CPU 6 (dev=26)
Adding CPU 6, hwref=26, awake=1
Waking up CPU 7 (dev=27)
Adding CPU 7, hwref=27, awake=1
Waking up CPU 8 (dev=14)
Adding CPU 8, hwref=14, awake=1
Waking up CPU 9 (dev=15)
Adding CPU 9, hwref=15, awake=1
Waking up CPU 10 (dev=16)
Adding CPU 10, hwref=16, awake=1
Waking up CPU 11 (dev=17)
Adding CPU 11, hwref=17, awake=1
Waking up CPU 12 (dev=c)
Adding CPU 12, hwref=c, awake=1
Waking up CPU 13 (dev=d)
Adding CPU 13, hwref=d, awake=1
Waking up CPU 14 (dev=e)
Adding CPU 14, hwref=e, awake=1
Waking up CPU 15 (dev=f)
Adding CPU 15, hwref=f, awake=1
SMP: AP CPU #3 launched
SMP: AP CPU #11 launched
SMP: AP CPU #13 launched
SMP: AP CPU #15 launched
SMP: AP CPU #14 launched
SMP: AP CPU #6 launched
SMP: AP CPU #1 launched
SMP: AP CPU #12 launched
SMP: AP CPU #4 launched
SMP: AP CPU #8 launched
SMP: AP CPU #9 launched
SMP: AP CPU #2 launched
SMP: AP CPU #5 launched
SMP: AP CPU #10 launched
SMP: AP CPU #7 launched
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
ada0: Serial Number JK11A8B9H82U0F
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors)
mmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
pass0 at ahcich1 bus 0 scbus1 target 0 lun 0
pass0: <Hitachi HDS722020ALA330 JKAOA3MA> ATA8-ACS SATA 2.x device
pass0: Serial Nmmu_radix_mapdev_attr(0x6016000010000, 131072, 0)
umber JK11A8B9H82U0F
pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
pass0: Command Queueing enabled
WARNING: WITNESS option enabled, expect reduced performance.
GEOM: new disk ada0
GEOM: ada0: the secondary GPT header is not in the last LBA.
GEOM: diskid/DISK-JK11A8B9H82U0F: the secondary GPT header is not in the last LBA.
WARNING: / was not properly dismounted
opal0: providing initial system time
failed to load CTF for kernel: 2
start_init: trying /sbin/init
uhub0: 8 ports with 8 removable, self powered
ugen0.2: <Aspeed USB Virtual Hub> at usbus0
uhub1 numa-domain 0 on uhub0
uhub1: <Aspeed USB Virtual Hub, class 9/0, rev 2.00/1.00, addr 1> on usbus0
Waiting 30s for the root mount holders: usbus0uhub1: 5 ports with 5 removable, self powered
ugen0.3: <OpenBMC virtualinput> at usbus0
usbhid0 numa-domain 0 on uhub1
usbhid0: <OpenBMC virtualinput, class 0/0, rev 2.00/1.00, addr 2> on usbus0
hidbus0: <HID bus> numa-domain 0 on usbhid0
hkbd0: <OpenBMC virtualinput Keyboard> numa-domain 0 on hidbus0
kbd: new array size 4
kbd1 at hkbd0
kbd1: hkbd0, generic (0), config:0x0, flags:0x3d0000
usbhid1 numa-domain 0 on uhub1
usbhid1: <OpenBMC virtualinput, class 0/0, rev 2.00/1.00, addr 2> on usbus0
hidbus1: <HID bus> numa-domain 0 on usbhid1
hms0: <OpenBMC virtualinput Tablet> numa-domain 0 on hidbus1
hms0: 3 buttons and [XY] coordinates ID=0
.ugen0.4: <vendor 0x05e3 USB2.0 Hub> at usbus0
uhub2 numa-domain 0 on uhub0
uhub2: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/32.98, addr 3> on usbus0
uhub2: MTT enabled
uhub2: 4 ports with 4 removable, self powered
.ugen0.5: <vendor 0x0d8c USB Sound Device> at usbus0
uaudio0 numa-domain 0 on uhub2
uaudio0: <vendor 0x0d8c USB Sound Device, class 0/0, rev 1.10/0.10, addr 4> on usbus0
uaudio0: Play[0]: 48000 Hz, 4 ch, 16-bit S-LE PCM format, 2x4ms buffer. (selected)
uaudio0: Play[0]: 44100 Hz, 4 ch, 16-bit S-LE PCM format, 2x4ms buffer.
uaudio0: Record[0]: 48000 Hz, 2 ch, 16-bit S-LE PCM format, 2x4ms buffer. (selected)
uaudio0: Record[0]: 44100 Hz, 2 ch, 16-bit S-LE PCM format, 2x4ms buffer.
uaudio0: No MIDI sequencer.
pcm0 numa-domain 0 on uaudio0
pcm0: Mixer "vol" -> "none": child=0x00000010
pcm0: Mixer "pcm": parent="vol"
pcm0: Mixer "line":
pcm0: Mixer "mic":
uaudio0: HID volume keys found.
uhub_explore: illegal enable change, port 1
Setting hostuuid: ce1c3c60-3bec-48a7-8c41-adc4cb68bbb1.
Setting hostid: 0xfc69d1ea.
Starting file system checks:
** SU+J Recovering /dev/ada0p2
** Reading 655589376 byte journal from inode 4.
** Building recovery table.
** Resolving unreferenced inode list.
** Processing journal entries.
/dev/ada0p2:
**** FILE SYSTEM MARKED CLEAN ****
/dev/ada0p1: 2834 files, 97 MiB free (197923 clusters)
FIXED
/dev/ada0p1: MARKING FILE SYSTEM CLEAN
Mounting local filesystems:.
Autoloading module: icee
KLD icee.ko: depends on kernel - not available or version mismatch
Setting up harvesting: PURE_DARN,RANDOMDEV,[CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: fsync: fsync /boot: Invalid argument
.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/glib-bootstrap/lib /usr/local/gobject-introspection-bootstrap/lib /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.42/mach/CORE
mixer: pcm2.volume=0.00:0.00: Device not configured
mixer: line1.volume=0.00:0.00: Device not configured
Setting hostname: freebsd-ppc64le-dev.
lo0: link state changed to UP
bge2: Disabling fastboot
bge2: Disabling fastboot
bge2: link state changed to DOWN
bge2: link state changed to UP
Starting Network: lo0 bge0 bge1 bge2.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 2c:09:4d:00:08:fa
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 2c:09:4d:00:08:fb
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge2: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 2c:09:4d:00:08:fc
        inet6 fe80::2e09:4dff:fe00:8fc%bge2 prefixlen 64 scopeid 0x3
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
Starting devd.
Starting Network: bge0.
bge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 2c:09:4d:00:08:fa
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Starting Network: bge1.
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 2c:09:4d:00:08:fb
        media: Ethernet autoselect
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Autoloading module: icee
KLD icee.ko: depends on kernel - not available or version mismatch
Autoloading module: icee
KLD icee.ko: depends on kernel - not available or version mismatch
Autoloading module: icee
KLD icee.ko: depends on kernel - not available or version mismatch
Starting dhclient.
DHCPREQUEST on bge2 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.140 -- renewal in 21600 seconds.
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Creating and/or trimming log files.
Clearing /tmp (X related).
Updating /var/run/os-release done.
Updating motd:.
Starting syslogd.
No core dumps found.
Starting powerd.
KLD mac_ntpd.ko: depends on kernel - not available or version mismatch
Starting ntpd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system checks in 60 seconds.

Tue Mar  3 02:28:14 CST 2020

FreeBSD/powerpc (freebsd-ppc64le-dev) (ttyu0)

login:
tpearson_raptorengineering.com added a comment.Wed, Feb 18, 3:40 PM

This is why I was saying we should probably find the fault in my patchset, and fix that, along with whatever is going wrong with the DMA tag setup where the mpr driver is allocating too much bounce memory. There's something different in the DMA core vs. what the old powerpc DMA code is trying to do, and whatever it is is quite subtle.

tpearson_raptorengineering.com added a comment.EditedWed, Feb 18, 3:49 PM

All that said, I bet if the mpr DMA issue was fixed the problem introduced in the port would at least be masked. I'm unclear what the "correct" behavior should be when we run out of bounce buffers; with the stock kernel, it looks like ahci and usb are simply denied buffer allocation (-ENOSPC) so they fall back to a DMA-free mode, whereas with the patch series applied they're (successfully) allocated too small of a bounce region and just lock up.

adrian added a comment.Wed, Feb 18, 3:54 PM

ok, does this happen without this domain change? can you just try D55312, D55313, D55314?

This domain change can and does change how physmem gets allocated. I can also go and experiment with setting up a couple separate sockets in my p8/p9 QEMU setup as right now i'm testing single socket, multi-core/thread topologies.

tpearson_raptorengineering.com added a comment.Wed, Feb 18, 4:01 PM

That actually does make sense -- physmem is different on PowerNV vs. pSeries, especially because use of the IOMMU is effectively mandatory on pSeries. I'm not sure you're going to be able to reproduce easily on qemu short of setting up a PowerNV system to test with, which in turn requires a current skiboot build, etc. Additionally, the mpr driver is unique in the entire tree as far as I can tell, I'm not sure the DMA over-allocation issue will be reproducible without an LSI card attached.

tpearson_raptorengineering.com added a comment.Wed, Feb 18, 4:16 PM

Seeing the same problem with just D54745, D55313, and D55314. I suppose it's possible D54745 could shift things around enough to make the problem appear, but I'm not comfortable merging code into the DMA system when I already know we're violating platform constraints (without D54745 I know for a fact we're allocating bounce buffer in an illegal region and Bad Things via undefined behavior are likely -- just because we get away with it long enough to boot doesn't mean we're not going to get data corruption and lockups later on).

adrian added a comment.Sat, Feb 21, 2:32 AM
In D55315#1266124, @tpearson_raptorengineering.com wrote:

Seeing the same problem with just D54745, D55313, and D55314. I suppose it's possible D54745 could shift things around enough to make the problem appear, but I'm not comfortable merging code into the DMA system when I already know we're violating platform constraints (without D54745 I know for a fact we're allocating bounce buffer in an illegal region and Bad Things via undefined behavior are likely -- just because we get away with it long enough to boot doesn't mean we're not going to get data corruption and lockups later on).

yup, i'm working on a subsequent diff that pulls in the rest of your work. i still though want to figure out the bounce buffer in illegal region thing first. That's just plain bad :-)

Revision Contents
Changeset List

PathSize
sys/
powerpc/
include/
1 line
powerpc/
5 lines
8 lines

Diff 172130

View Options

sys/powerpc/include/bus_dma_impl.h

Loading...
View Options

sys/powerpc/powerpc/busdma_bounce.c

Loading...
View Options

sys/powerpc/powerpc/busdma_machdep.c

Loading...