sendfile: Fix bug when using headers with SW KTLS offload

[gallatin]

Summary

When using SW KTLS, we must account for the headers in sf_iodone() in terms of either freeing or enqueuing them for TLS work. Not doing so can lead to a situation where we enqueue only the payload, and not the header, for encryption. Rather than leaking the header, the socket is left "hung" with the header marked M_NOTREADY.

We noticed this at Netflix when working on an SSL_Sendfile that allows using the hdtr feature of sendfile.

[kib]

sys/kern/kern_sendfile.c

346

Why 'else'? sfio->m could point to m0 AFAIS.

[gallatin]

sys/kern/kern_sendfile.c

346

I'm not sure I understand. The mbuf chain is constructed as such that the final "m0" is the first payload mbuf, and "m" is the header mbuf, which then points to m0 as next.

So with header, we have: mh->m0....mtail
and without header we have: m0...mtail

So if we omit the else, then we ready m0..mtail twice.

[kib]

sys/kern/kern_sendfile.c

346

I was asking to understand.

[gallatin]

sys/kern/kern_sendfile.c

346

I view you as so much better at seeing edge cases and tricky interactions than me that when you ask a question like that, I'm assuming that I am the one who has missed something important. :)

[glebius]

sys/kern/kern_sendfile.c

346

<offtopic>
When a Westerner asks a question, they point at a mistake. When Slavic guy asks a question, he is actually asking a question to fulfill his knowledge.

Myself trying to balance between the two communication cultures is on a brink of mental disorder :)
</offtopic>