Make mmap fail when len parameter, round up to page size, becomes zero

[dougm]

Summary

There are times when a len==0 parameter to mmap is okay. But on a 32-bit machine, a len parameter just a few bytes short of 4G, rounded up to a page boundary and hitting zero then, is not okay. Return failure in that case.

Test Plan

Peter Holm reported this problem, and reported that this change resolved it.

[kib]

vm_mmap.c

261

Why do you need the cast ? size already has size_t type, and vm_size_t is effectively an alias to it.
And if cast is needed, there should be no space after ')'.

[dougm]

Drop needless casts.

[alc]

I think that it is likely that other, related system calls exhibit a similar misbehavior.