powerpc64/pmap: Reduce scope of PV_LOCK in remove path

[jhibbits]

Summary

Since the 'page pv' lock is one of the most highly contended locks, we
need to try to do as much work outside of the lock as we can. The
moea64_pvo_remove_from_page() path is a low hanging fruit, where we can
do some heavy work (PHYS_TO_VM_PAGE()) outside of the lock if needed.
In one path, moea64_remove_all(), the PV lock is already held and can't
be swizzled, so we provide two ways to perform the locked operation, one
that can call PHYS_TO_VM_PAGE outside the lock, and one that calls with
the lock already held.

[nwhitehorn]

Looks good to me; do you know how much this actually helps?

[luporl]

Nice, LGTM.

[jhibbits]

In D20694#447179, @nwhitehorn wrote:

Looks good to me; do you know how much this actually helps?

Not much, unfortunately. Might give us a ~1-2% improvement on buildworld, but that's it. We really need to relook at moea64_enter(). The page pv lock is held way too long in there, and there's a lot of contention on it when a single page is mapped into multiple pmaps. Can you take a look at that bit? My naiive approach to fixin gmoea64_enter() seemed to make things significantly worse instead.