Special handling for -o, open the fd before entering the sandbox, then set the mode on it later (the intended mode is specified in the input)
Just some high level comments. I think if the diff is smaller, it's easier to review the rest.
Why have this at all?
We keep a dirfd around which can open outfile anyway — no need to open the file before entering sandbox.
It duplicates the safe-output-file checking logic below, except for potential time-of-check vs time-of-use attacks. IMO, this whole function can be removed.
I think in general we should preopen an array of fds rather than FILE objects, then fdopen as needed. (The concern is that the overhead of FILE objects vs fds is meaningful.)
One potential cost is the size of the stdio buffering. I guess I would be a little shocked if FILE prefetched file contents or allocated much memory on open. But even so a FILE is bulkier than just an fd.
I think you could avoid a lot of unnecessary code churn of this patch by setting globals infile, infp, (as is the existing pattern in this code) rather than passing these things as parameters.