Is all this trickery to reduce sgl lengths really worth it for what's going to be 4K chunks? That's the largest PCIe max read request size anyway, so the device is going to have to split the segments that you've combined regardless. All we're doing here is maybe compressing the SGL,s but it just doesn't seem worth the effort to me.

I think the tradeoff is different here than in what might be a series of 128b mbufs written on a socket, for example.

Shouldn't this be ktls_toe_{cbc,gcm} ?

Hmm. Ironically pretty much all the bugs I had were in this function, though they weren't due to dealing with coalescing (and at this point they are all resolved). The other interesting thing is that the main driver also tries to coalesce ext_pgs mbufs as well as the NIC TLS code, and arguably it would simplify all of those to not bother with coalescing. I'll talk to np@ to see what the tradeoff is for coalescing SGL entries and what effect that might have (e.g. it might also use less resources on the NIC itself for all I know to store the more compact SGL which might matter when you have a lot of active connections)

Yes, I fixed this yesterday when i noticed the counters were wrong and need to update the diff.

Compressing SGLs so that they use fewer tx descriptors is definitely worth it for any TOE txq (ulp mode crypto or not). The the per-tid toep->tx_credits are in high demand during tx and it makes sense to pack as much as possible into as few len16 tx credits for bulk transmit.

Should be 'tls_new->mode'. Will fix.