oh crap i missed testing this; lemme test this on -HEAD today on the G5 and POWER8.

• convert the superpage lock path to use the iterator lock
• and another function involved as well

delete old code

updated; i removed the recursive lock as now we shouldn't be recursing.

update

not needed now!

Committed in 1364e7d0921b

In D55095#1259555, @tpearson_raptorengineering.com wrote:

In D55095#1259290, @adrian wrote:

oh that's a good catch!

What about regions, pregions, numa_pregions, etc? Are there any other shared variables that need to be unified here?

I checked; the only function we use from the HPT side of things is moea64_bootstrap_alloc(), therefore this is the only variable that needs to stay in sync. While I can't say I particularly like this brittle architecture, I suspect it's going to be changed so infrequently that it's not worth rewriting everything at this point.

LGTM. Good find! Approved.

looks good, land when you're ready!

In D55095#1259290, @adrian wrote:

oh that's a good catch!

What about regions, pregions, numa_pregions, etc? Are there any other shared variables that need to be unified here?

After overnight stress testing, a combination of D55095 and using the correct DMA tag seems to have completely resolved the AHCI instability I was seeing with this patchset applied.

Interesting, so mmu_radix depends on mmu_oea64 for bootstrap allocations, which in turn depends on its own phys_avail_count. Can you just put the extern declaration in mmu_oea64.h instead of having the extern in mmu_radix.c? mmu_radix.c already includes mmu_oea64.h, so that looks like the best place to put these shared variable declarations.

I think this is fine; let's see what justin thinks tomorrow morning!

oh that's a good catch!

Thanks for that!

Also /proc/iomem, which i think has what you're after in more specific detail:

here's petitboot:

Digging further, at least some of the problem here seems to be from our rather unique memory allocation on POWER9:

OK, thanks for testing. I spent most of the day already going over the codebase trying to figure out what might be going wrong, and all I've come up with thus far is that the 32-bit MMIO window setup and DMA configuration both make no sense. It shouldn't even be working at all on POWER9 using the stock code (without this patch).

ok, two normal boots and then the boot with this diff attached are at https://people.freebsd.org/~adrian/powerpc64/20260203-D54745-fail-1.txt .

so it didn't finish booting;

ok, its definitely unhappy, stay tuned!

rebase after jhibbits@ recent changes

! In D54745#1259084, @adrian wrote:
my POWER8 box does, but that may not be good enough?

In D54745#1258964, @tpearson_raptorengineering.com wrote:

When testing, does anyone else have access to a SATA controller that does DMA? I'm sporadically seeing the Blackbird's AHCI controller lock up but I don't know if this is a PE freeze, bad DMA, or something completely unrelated (flaky cabling?):

ahcich1: is ffffffff cs ffffffff ss ffffffff rs ffffffff tfd ffffffff serr ffffffff cmd ffffffff
(ada0:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 40 e8 35 8b 40 24 00 00 00 00 00
(ada0:ahcich1:0:0:0): CAM status: Command timeout
ahcich1: AHCI reset...
(ada0:ahcich1:0:0:0): Retrying command, 3 more tries remain
ahcich1: stopping AHCI engine failed
ahcich1: SATA connect timeout time=100000us status=ffffffff
ahcich1: AHCI reset: device not found
pass0 at ahcich1 bus 0 scbus1 target 0 lun 0
pass0: <Hitachi HDS722020ALA330 JKAOA3MA> s/n JK11A8B9H82U0F detached
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <Hitachi HDS722020ALA330 JKAOA3MA> s/n JK11A8B9H82U0F detached
g_vfs_done(): ada0p2 converting all errors to ENXIO
g_vfs_done():ada0p2[WRITE(offset=212411777024, length=32768)]error = 6 suppressing further ENXIO
panic: UFS: root fs would be forcibly unmounted

@adrian Could you give this patch set a thorough test as well? It is not dependent on D54745, I mainly want some additional testing without this on top of D54745 and a merge before going to the next phase of IOMMU enablement. D54745 has some issues with DMA and the AHCI controller that I don't currently understand and will need further investigation.

When testing, does anyone else have access to a SATA controller that does DMA? I'm sporadically seeing the Blackbird's AHCI controller lock up but I don't know if this is a PE freeze, bad DMA, or something completely unrelated (flaky cabling?):

Do It!

(I still haven't forgotten about this diff; I'm going to test it in VMs and on power8 hardware this week.)