This is more of a RFC than a commit candidate. When I tried to use /dev/mem to access a register I got a "system error abort" panic (but mapping the same memory region with new-bus as a struct resource worked fine). The use case for me was using dd if=/dev/mem iseek=<register physical address/4> bs=4 count=1 | hd is how I got the SERR# panic. I'm not sure if the difference is that I should be using a different memattr perhaps? On amd64, this code uses pmap_kenter for the !can_fault case to avoid TLB shutdowns. I'm not sure that is as important for arm64 since shutdowns are presumably a bit cheaper which is why I did the lazy thing.

See also the discussion in D47678 from when I implemented what looks like the same patch

I think those issues are not made any worse by your patch? And the fix is something that RISC-V and amd64 would need as well (well, RISC-V also needs this patch). I think you should just merge your current patch and let @markj fix both amd64 and aarch64 when he adds the new arena.

In terms of the pinning vs not-pinning, you could indeed drop the pinning I guess if you use pmap_qenter() always (which I think is what alc@ was suggesting).