libutil: take a size_t in trimdomain()

[kevans]

Summary

INT_MAX is already larger than a reasonable hostname might be, but
size_t makes some of this easier to reason about as we do arithmetic
with it. This would maybe not be worth it if we had to bump the
soversion because of it, but libutil does symbol versioning now so we
can provide a compat shim.

While we're here, fix some inconsistencies in argument names in the
manpage.

Obtained from: https://github.com/apple-oss-distributions/libutil
Sponsored by: Klara, Inc.

[ziaee]

lib/libutil/trimdomain.3

66–67

What's an nn?

[jrtc27]

lib/libutil/trimdomain.3

66–67

Integer display and screen numbers in an X11 DISPLAY env var

[des]

lib/libutil/trimdomain.c

96

should you check that hostsize is nonnegative? Clamp it to zero perhaps?

[kevans]

lib/libutil/trimdomain.c

96

I had considered it, and I'm still a little torn. Here was my thought process: existing programs passing in a negative size today would end up with the following consequences:

1. end <= (s = fullhost)
2. (size_t)(end - s) is then a large size_t value

Thus it has a chance of working, but still the chance of overread addressed in the next patch. Clamping it to 0 means that it can't be successful even for a fullhost with a valid format, so I concluded that it might be better to just let it promote / convert to unsigned.