fcntl(2): be more precise about third arg type

Also use the term operation consistently, over the command.

Reviewed by: emaste, jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33277