Fix build for LP64 arches with gcc.

gcc complaints that the comparision is always false due to the value
range, and the cast does not prevent the analysis. Split the LP64
vs. ILP32 clamping as a workaround.

Sponsored by: The FreeBSD Foundation