tarfs: Fix support for large files

• When fast-forwarding through a zstd frame, we incorrectly used the min() inline function instead of the MIN() macro. The function truncates the result to unsigned int, resulting in a decompression error when trying to seek more than 4 GB into the frame.

• POSIX states that a size extended header record overrides the size field in the header if present, and that one must be included if the size of the file exceeds 8 GB (the size field maxes out at 64 GB).

• Reduce repetition in the exthdr parser by deduplicating the syntax error handler.

MFC after: 1 week
Sponsored by: Klara, Inc.
Fixes: 69d94f4c7608 ("Add tarfs, a filesystem backed by tarballs.")
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D53718