vmem: Fix the gcc build

gcc complains when building libuvmem because the last two operands of ?:
in ORDER2SIZE and SIZE2ORDER have different signs. Add explicit casts
to address this.

Reported by: Jenkins
Reviewed by: alc, kib
MFC after: 1 week
Fixes: 1ecf01065b45 ("libuvmem: usermode port of vmem(9)")
Differential Revision: https://reviews.freebsd.org/D54268