cxgbe tom: Set the tid in the work requests to program page pods for iSCSI.

[jhb]

Summary

As a result, CPL_FW4_ACK now returns credits for these work requests.
To support this, page pod work requests are now constructed in special
mbufs similar to "raw" mbufs used for NIC TLS in plain TX queues.
These special mbufs are stored in the ulp_pduq and dispatched in order
with PDU work requests.

[jhb]

sys/dev/cxgbe/tom/t4_cpl_io.c

1113

I'm not 100% certain of the reason for the change from 3/8 to 5/8 here, and the new comment above probably could use some tweaking? It seems like the new comment instead means something like forcing F_FW_WR_COMPL anytime toep->tx_credits < SGE_MAX_WR_LEN via a new condition rather than the 3/8 -> 5/8 change?

[np]

sys/dev/cxgbe/tom/t4_cpl_io.c

1113

tx_total is 84 with default fw config so 3/8 of 84 is 31. These are len16 so that's 31 x 16 = 504B, which is less than SGE_MAX_WR_LEN. The updated driver probably sends a lot more full sized work requests for writing page pods so the 3/8 can cause tx flow control issues between driver/firmware.

1/2 would have worked, 5/8 works too. But they depend on tx_total being 84, which is not guaranteed. A more flexible test would be what you suggested, something like this:
if (toep->tx_credits <= SGE_MAX_WR_LEN * 5 / 4)

[jhb]

• Update condition and comment for requesting credit return.