Add the hw.bnxt_tx_db_coalesce tunable (default 1, maximum 128). Above 1, up to that many flushes share one doorbell, and bnxt_if_timer()
rings what is left once a queue has been idle for a tick, so no descriptors are stranded after a burst. The default of 1 keeps the current behavior.
Details
Testing in the Broadcom labs indicates that setting bnxt_tx_db_coalesce=64 yields optimal UDP performance on the Broadcom Thor2 NIC.
Diff Detail
- Lint
Lint Skipped - Unit
Tests Skipped
Event Timeline
If I'm reading this right, a lone packet waits for two timer callbacks. bnxt_isc_txd_flush() saves the producer index and increments tx_db_seq without ringing. The next bnxt_if_timer() seees a changed sequence and only records it; the following callback rings if the sequence stayed unchanged. The default iflib period is hz / 2 (sys/net/iflib.c:6293), so this adds approximately 500 ms–1 second per affected transmit on an idle queue.
Other issues are that this is safe only for simple_tx (which is fine for me, but not if its MFC'ed) since simple tx uses the same mtx as the timer, but mp_ring doesn't.
Doorbell batching should respect iflib’s forced flushes and flush when a TX batch drains. Any timed deferral needs an explicit, short latency bound and must run under the same serialization as TX; its state must also be reset with the hardware ring.