Limit RESOLUTION_MAX to INT_MAX, not UINT_MAX (all spelled out) so the
mode value isn't always clipped to -1 when (resolution * size) == 32, which
would have been the case with values => {4i,32b,32t}.

This seems to have been broken in r64382.

MFC after: 1 week
X-MFC with: r289915
PR: 200619
Reported by: Michael Baptist
Submitted by: Lars Skodje
Sponsored by: EMC / Isilon Storage Division