The bnxt driver was using the "inner" IP and TCP/UDP
checksum fields when populating the mbuf's checksum
offload fields. That is fine when the inner frame
is an IP+TCP/UDP frame, since it is implied that
the inner checksum will only be examined if the "outer"
VXLAN checksums are Ok.
However, if the inner payload is not IP+TCP/UDP,
i.e. Tidalscale cluster traffic, then the inner
checksum won't be computed, and the mbuf won't
have any checksum information attached even though
the VXLAN UDP tunnel checksum is valid. This results
in the decapsulation code having to perform an
unnecessary s/w checksum.
Fix by checking both the inner and tunnel checksum
results. It isn't possible for the inner checksum
to pass if the outer one doesn't, so it's fine
to check both at one, which will cover both the
tunnel+non-IP and no-tunnel cases.