I suggest we add statistics/counters for that particular situation. It'd be good to track it down and check if all hw is prone to this issue or it's only one which Ed was using.

Any feedback?

Any other comments?

Thanks. In that case I'll wait for the generic minidump format to be integrated first.

Removed the first check.

The goal for this thing is to have the USB controller (sys/boot/usb & sys/boot/kshim library) independent of u-boot.

Ok, I see. I'll try to change the code and use pmap_kextract to check which pages are mapped to avoid parsing of pagetables directly. However I still will need to access some of PMAP structures to prepare fake-pagetables for libkvm... Please let me know if that approach would be acceptable. I don't want to spend much time doing that and then find that everything is wrong :)

Thanks, eventually that'd be nice. However I'd like to submit this patch and then remove any old-pmap stuff in a separate commit. It's just that we have prepared new-pmap to be MFCed to 10-stable and I'd like to have every necessary change isolated to keep the track of all required commits.
If you have no objections I'd like to ask zbb to commit this change as is.

I don't understand how is your question related to this change. Especially that I do use loader...

Ok, thanks for all comments. So... what are we going to do with this patch in general?
I assume that it might be impossible to avoid libkvm+minidump from using PTE tables directly. Yes, we can use a pmap_kextract in minidumps to check if a given page is mapped to the kernel, but we'd still need to create a "fake" pagetables to be compatible with libkvm. What do you think?

I removed all ifdefs to avoid obfuscating the code with reserved fields etc.

Argh... I must have been drunk I didn;t notice the parsing function is already there :) With few tweaks it looks usable.

Hm, are you sure about this ? Shared object's text is mapped private, same as the main binary text. The proc_rwmem() functionality, which backs ptrace_write_int(), correctly handles COW, or, at least it did when I last checked it.

Yep, you're right. Currently the ptrace stepping works only for very special case. The interface requires redesign, but I'd like to have this patch integrated first, just to let some of debugging be working.
There is also a problem, when you try to step over the code from a shared library. Modification of any instruction becomes global for all processes using the same lib.
However, I believe that someone trying to use this functionality now is aware of some of its limitations. Or at least should be...

Good point, John. That would also explain why sriov-capable cards are working fine and do not require this hack. I'll look at ari configuration more closely.

I'm fine with that workaround. Also, don't think the configuration variable is necessary. I've just tested few cards and the em is not the only one prone to multi-enumeration. Let's leave it as is, we'll only loose the ability of using pcie switches in thuderx external pci, which is in fact an extremely rare scenario.
This week I'll try to at least check the generic pci to ensure it properly configures thunderx internal bridge.

Fixed everything except sb_zbpci, as you suggested.

Yep, that's also the way we can do this.

39 bits are the maximum address range the MMU can handle using 4KB pages and L1&L2&L3 pagetables. This corresponds to mapping of 512GB of VA.
Architecturally ARM64 can use up to VA 48-bits but that would require either using L0 pagetable as well (thus provide way more complex address translation and slow down the whole system, not only dmap pa/va resolving) or increase page size to 16KB or 64KB (that is also undesireable).

The problem is, that on ARM64 we do not have physically continuous memory. If we take ThunderX as an example, there is:
0#64GB @ 0x00000000_00000000 - 0x0000000F_FFFFFFFF
1#64GB @ 0x00000100_00000000 - 0x0000010F_FFFFFFFF
EFI runs us from region 1#, so we must have this mapped, because there is also where L1 pagetable is located.

Fix:

• PA chunks are 1GB-aligned
• fix minidumps

Any comments?

Andrew, do you have any comments? If not, i'd like to submit this change and close the review.

We found some troubles with CPU starting on one of our non-cavm arm64 boards. If you have no objections I'd like to submit this patch tomorrow.

As discussed with Ed, a simpler temporary workaround will be added to gicv3.

Thanks, good point!
The next step for this driver is to enhance resource allocation in case the size is unknown (line #442) which is required for ThunderX ethernet NIC we're currently developing the driver for.
If you don't mind, I'd like to commit this file as-is and add support for NEW_PCIB later, along with previously mentioned VFs. I think I should make this in a week or two.

Applying fixes from internal PCIe review here also.

Marking all issues as done.

Cosmetic changes: aligned with the current driver version to ease any further merge of external pcie driver.

I like the idea of doing sth generic. What is your ETA on this?
If it takes more than 2-3 weeks maybe it'd be good to submit anything that gives us support for PPIs and revert it once an elegant solution is done? I mean this patch or even port an ugly hack from gic.c, whatever, since it'd be only temporary. Without this, the GICv3 combined with SMP is pretty useless.
Andrew, Ed, what do you think?

Added current info about Cavium revisions. Let me know if that's enough.

Removed all fdt-related stuff and cleaned up a little.

Andrew, would you mind explaining what is the problem here?
At first you disliked an idea of having any vendor-specific defines in kernel config (option SOC_CAV_THUNDERX) so I developed a method for resolving it dynamically in runtime. That approach is really optimal and gives us only 5-cycle penalty when the cpu is not thunderx (reading data from pcpu takes 3-cycles if is in L1 cache, value comparison and near jump are another 2-cycles). That's way better than workarounds in Linux, and, what's more, marginal when you compare with cycles spent in the PIC interrupt handler routine.
Now we are adding magical errata option to kernconf and creating mess, which we'd like to avoid... But that's fine if that's what you want. My primary goal is to have thunderx support done.

I mostly dislike an idea of having errata-specific ifdefs inside the code. This one is the first, but there are still few more I'd like to submit and would like to avoid causing mess within generic drivers.