It looks great now! Since this changes core libc functions, can you perform a buildworld on a system with the modified libc installed to confirm everything works fine?

@bruno.larsen_eldorado.org.br, overall your changes look good to me.
I have only one more suggestion and some nitpicks.

In D28776#646046, @alfredo wrote:

In D28776#645964, @bruno.larsen_eldorado.org.br wrote:

• improved formatting
• solution to the problem outlined by luporl

The alignment problem is solved, but this patch makes bcopy performance go down by a factor of 10. I think a third version of the code should be considered, restricting this patch only to POWER7 CPUs.

Does it produce any gain over original non-optimized code on Power7? If there's no gain I'd suggest restrict optimizations to ISA >= v.2.07 (POWER8). And in this case, remove unaligned access handling since newer CPUs can handle it transparently.

If there's still a gain I agree with having two optimized versions one handling VSX & ISA <=2.06 and the other for VSX & ISA >=2.07 (probably removing unaligned handling to make code simpler if no negative performance impact)

Worked fine on PPC64LE, both on Talos and QEMU, with D27475 to fix boot with ofwfb on LE.

Looks good. I have been using a slightly modified version of this on Talos II and it is working well.

Using only htole64() won't work in all cases.

The changes look good to me overall.

Leave superpages disabled by default

Wow, that's really a lot of endianness fixes, nice job!

Restore comment about tlbie instruction forms

• improve performance
  • force inline in new moea64_pte_* functions to avoid extra overhead with clang -O now defaulting -O1
  • look for superpage PVO in moea64_enter only when superpages are enabled
  • use multiple PV locks for superpage ops, instead of making a single PV lock cover a superpage

Rename ppcpnv/powernv to power8.

This diff has several changes on top of the previous one. The main ones are:

Fix ERAT multi-hit issue

Looks good to me.

• address (part of) markj's comments

Address jhibbits' comments

Address bdragon's comments

In D26114#580412, @bdragon wrote:

Makes sense to me. Is there a good way to test this?

Additionally, I believe this would also be relevant to ppc32, perhaps use the __powerpc__ ifdef instead.

In D26113#579628, @kbowling wrote:

One thing we do have is the full power8/9 pmcs in src/lib/libpmc/pmu-events/arch. I think these just need to be hooked up

Looks good to me.

Looks good, very similar to amd64 code.
But if you want to make it equivalent to amd64 code, you could also print SP, PC (that would be frame->fixreg[1] and frame->srr0, I think) and the word (or maybe the first 2 words) at frame->srr0.

Nice. I just have a few more nitpicks.

Just for the record, measuring buildworld (as well as kernel and libc builds) times with superpages enabled and disabled didn't show any significant variation.
But when comparing the build times with a kernel that has D25237 vs one without it, there is a consistent performance hit of about 3%.

• Address jhibbits comments

Moved PVO_PADDR() changes to another diff (D25654).

Address kib's review comments

In D25550#564939, @kib wrote:

Provide the amd64 example, in binary form.

In D25237#563281, @markj wrote:

As an intermediate first diff it would be nice to split out the changes to use PVO_PADDR().

Do you have an idea where the slowdown is coming from with pg_ps_enabled=1?

Right now I'm planning to profile the code to try to improve superpages performance, that should at least present some gain over not using it.

• Add more specific sysctl counters about promotion failures.
• Add an early check to handle a very common promote fail case, that dramatically reduces the number of promotion failures and improves buildworld performance substantially.
• Factor out different PPC64 tlbie instruction forms into an ifunc, which also corrects large page invalidation on older cores.

• Address jhibbits' comments

• add missing count increment when handling superpages, in moea64_clear_bit()