wtap: use typed rssi and noise floor values

Adjust the rssi and nf arguments to typed int8_t and adjust the maths
for rssi to be consistant with what net80211 expects.

Sponsored by: The FreeBSD Foundation
Reviewed by: lwhsu, adrian
Differential Revision: https://reviews.freebsd.org/D57020

(cherry picked from commit d201e4e8491901d1de6bcaeb581a0bf958bf86f2)