vm_object_coalesce(): return swap reservation back if overcharged

[kib]

Summary

It is possible for both vm_map_insert() and vm_object_coalesce() to charge
both for the same region.  The issue is that vm_map_insert() must charge
in advance to ensure that the mapping would not exceed the swap limit,
but then the coalesce might decide to extend the object, and already
(partially) backs the mapped region.

Handle this by passing to vm_object_coalesce() exact information about
the charging mode of the extending range 'not charging', 'charged' using
flags instead of simple boolean.  In vm_object_coalesce(), detect
overcharge and undo it if needed.

Note that this relies on vm_object_coalesce() call being the last action
in vm_map_insert() before extending the previous map entry.

Reported by:    pho

[kib]

NB: this is still being tested by Peter, I decided to go ahead and publish the review as an early exposure

[pho]

I ran the full set of stress2 tests, without seeing any issues.