math(3): Use the .Fa macro for function arguments

.Fa is the suitable macro for functions in comparsion to the
.Ar macro, which should be used for commandline arguments.

While here, fix some mandoc warnings.

Reviewed by: imp (earlier version)
Obtained from: OpenBSD (in partial)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D31090