Page MenuHomeFreeBSD

cxgbe: Flush transmitted packets more regularly in netmap mode
Needs ReviewPublic

Authored by markj on Nov 7 2018, 7:54 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, May 22, 8:50 AM
Unknown Object (File)
Sun, May 19, 10:58 AM
Unknown Object (File)
Sun, May 5, 3:11 AM
Unknown Object (File)
Fri, May 3, 8:04 AM
Unknown Object (File)
Fri, May 3, 7:50 AM
Unknown Object (File)
Fri, May 3, 5:32 AM
Unknown Object (File)
Apr 8 2024, 11:28 AM
Unknown Object (File)
Dec 20 2023, 1:23 AM
Subscribers

Details

Reviewers
np
bwicht_verisign.com
Group Reviewers
Klara
Summary

Previously, when transmitting short runs of packets via cxgbe_nm_tx(),
we would wait until a large number of packets were buffered before
scheduling a task to clean transmit buffers.

Obtained from: np

Diff Detail

Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 20676
Build 20091: arc lint + arc unit

Event Timeline

@jhb Can you take a look at this?

@brd what kind of workload do you see the improvements with?

There are no updates here but I discussed this (and a couple of other t4_netmap.c changes) with the author and submitter at vBSDCon 2019 and we thought it best to leave this as a private change in their repo at that time.

If this helps other users too then we could replace the on/off style lazy_tx_credit_flush with a threshold at which to flush the credits instead of hardcoding it to 64.

In D17883#869748, @np wrote:

@brd what kind of workload do you see the improvements with?

I believe Brad was just hoping to see this committed on behalf of the submitter. Apparently this patch has been used in production for several years now.

There are no updates here but I discussed this (and a couple of other t4_netmap.c changes) with the author and submitter at vBSDCon 2019 and we thought it best to leave this as a private change in their repo at that time.

If this helps other users too then we could replace the on/off style lazy_tx_credit_flush with a threshold at which to flush the credits instead of hardcoding it to 64.

Is your suggestion to add a new lazy_tx_credit_flush_thresh sysctl, used as follows:

1033                 if (npkt == 0 && npkt_remaining == 0) {                                                                                                                                                                                                                                                                  
1034                         /* All done. */                                                                                                                                                                                                                                                                                  
1035                         if (lazy_tx_credit_flush == 0) {                                                                                                                                                                                                                                                                 
1036                                 wr->equiq_to_len16 |= htobe32(F_FW_WR_EQUEQ |                                                                                                                                                                                                                                            
1037                                     F_FW_WR_EQUIQ);                                                                                                                                                                                                                                                                      
1038                                 nm_txq->equeqidx = nm_txq->pidx;                                                                                                                                                                                                                                                         
1039                                 nm_txq->equiqidx = nm_txq->pidx;                                                                                                                                                                                                                                                         
1040                         } else if (NMIDXDIFF(nm_txq, equeqidx) >=                                                                                                                                                                                                                                                        
1041                             lazy_tx_credit_flush_thresh) {                                                                                                                                                                                                                                                               
1042                                 wr->equiq_to_len16 |= htobe32(F_FW_WR_EQUEQ);                                                                                                                                                                                                                                            
1043                                 nm_txq->equeqidx = nm_txq->pidx;                                                                                                                                                                                                                                                         
1044                         }                                                                                                                                                                                                                                                                                                
1045                         ring_nm_txq_db(sc, nm_txq);                                                                                                                                                                                                                                                                      
1046                         return;                                                                                                                                                                                                                                                                                          
1047                 }

?

markj added a reviewer: bwicht_verisign.com.
markj retitled this revision from cxgbe netmap: Directly flushes batches of 64 packets in lazy_tx_credit_flush to cxgbe: Flush transmitted packets more regularly in netmap mode.
markj edited the summary of this revision. (Show Details)

Rework a bit to reduce duplication.