The vm_pager_page_unswapped() pager op is only implemented for the
swap pager. Swap pager uses a private mutex to protect swap metadata,
and does not rely on the vm object lock to ensure integrity of it.

Weaken the requirement for the vm object lock by only asserting locked
object in vm_pager_page_unswapped(), instead of locked exclusively.

Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week