Page MenuHomeFreeBSD

stallamr_netapp.com (Sai Rajesh Tallamraju)
User

Projects

User Details

User Since
Nov 9 2020, 10:16 PM (31 w, 4 d)

Recent Activity

Yesterday

stallamr_netapp.com added a comment to D30733: ixl(4): Fix reporting of unqualified transceivers.

So, thinking on this, my guess is that when you reboot the machine, we would be finding an "unqualified" for a qualified cable because FW see this as link down. Also, my guess, the cable will show up as unqualified when you shut the link on link-partner.

I'm testing both scenarios with this transceiver:

plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
vendor: Intel Corp PN: AFBR-703SDZ-IN2 SN: AD1432A0AY9 DATE: 2014-08-11

and I don't see the unqualified message in the dmesg

Shutting down a link on the link partner does not affect reporting by FW if module is qualified.

Fri, Jun 18, 8:13 PM

Wed, Jun 16

stallamr_netapp.com requested changes to D30733: ixl(4): Fix reporting of unqualified transceivers.

I have applied the patch and rebooted my machine and I see "unqualified" message for a good cable.
The good thing is that with this patch I donot see "unqualified" message for a admin link-down.

Wed, Jun 16, 3:02 PM

Tue, Jun 15

stallamr_netapp.com added a comment to D30758: Check for non-empty sysctl_ctx before attempting free.

Thanks Krzysztof , will hold-onto this review and wait on you for further direction.

Tue, Jun 15, 2:27 PM

Mon, Jun 14

stallamr_netapp.com added a comment to D30733: ixl(4): Fix reporting of unqualified transceivers.

So, thinking on this, my guess is that when you reboot the machine, we would be finding an "unqualified" for a qualified cable because FW see this as link down. Also, my guess, the cable will show up as unqualified when you shut the link on link-partner.

Mon, Jun 14, 8:59 PM
stallamr_netapp.com accepted D30733: ixl(4): Fix reporting of unqualified transceivers.

Thanks Krzysztof.

Mon, Jun 14, 4:27 PM

Sun, Jun 13

stallamr_netapp.com added reviewers for D30758: Check for non-empty sysctl_ctx before attempting free: NetApp, krzysztof.galazka_intel.com.
Sun, Jun 13, 8:15 PM
stallamr_netapp.com requested review of D30758: Check for non-empty sysctl_ctx before attempting free.
Sun, Jun 13, 8:14 PM

Fri, Jun 11

stallamr_netapp.com added a comment to D30733: ixl(4): Fix reporting of unqualified transceivers.

Also, in original change, https://reviews.freebsd.org/D28028, I noticed that after executing ixl_set_link(pf, false), the PHY capabilities query for an_info has I40E_AQ_QUALIFIED_MODULE unset. So, the same supported/qualified module becomes unqualified.
I think the crux of the problem is with ixl_set_link() unsetting I40E_AQ_QUALIFIED_MODULE.

Fri, Jun 11, 5:05 PM

May 10 2021

stallamr_netapp.com added a comment to D30155: ixgbe: Bring back accounting for tx in AIM.

Also, I prefer to have a quick call and discuss the ideas & thoughts we have. We would need an expert from Intel to help us understand AIM.
On side note, from NetApp performance experiments on NetApp platforms, BSD11 (legacy driver) vs BSD12 (IFLIB base driver) - we noticed almost 3.5x-4x latency spike in one of write tests for IFLIB-based drivers.

May 10 2021, 2:11 PM
stallamr_netapp.com added a comment to D30155: ixgbe: Bring back accounting for tx in AIM.

I have similar observation (bad news) wrt UDP. But for TCP, I see just fine. My runs are all on NetApp platform.
Please note: my client is not HoL.

May 10 2021, 2:04 PM
stallamr_netapp.com added inline comments to D30155: ixgbe: Bring back accounting for tx in AIM.
May 10 2021, 1:25 PM

May 7 2021

stallamr_netapp.com added inline comments to D30155: ixgbe: Bring back accounting for tx in AIM.
May 7 2021, 3:27 PM

May 6 2021

stallamr_netapp.com added a comment to D30094: ixgbe: Enable AIM by default.

Thanks Kevin. Your observations seem to fall inline with my code understanding. I have been digging into this to understand performance impact, NetApp is having after migrating to IFLIB based drivers. Here are my observations so far (mainly with respect to Intel 10G/40G drivers):

May 6 2021, 7:57 PM

May 5 2021

stallamr_netapp.com requested review of D30094: ixgbe: Enable AIM by default.

Thinking more on this, lets cancel/ignore this change. In other words, keep the default setting to FALSE. Anyone who wants AIM, can enable on their platforms.

May 5 2021, 1:42 AM
stallamr_netapp.com commandeered D30094: ixgbe: Enable AIM by default.

Summarizing, this AIM will basically decide the value of ITR. Without AIM, ITR value will always stay at 128. With AIM enabled, ITR value now juggles between 32 - 8192. Below is the histogram on NetApp platform (12 processors, Intel Xeon D-1557 @1.50GHZ) when I had iperf3 ran for 1min.
I donot know exactly how ITR value translates to Interrupt moderation but to best of my knowledge, this value determines a minimum gap between two consecutive interrupts. From below histogram you can see, that, with more frequent data, ITR value concentrates more around 32 and 128.
That also implies, the gap between interrupts is lower when compared to AIM disabled i.e, 128. This falls inline with your observation too.

May 5 2021, 1:32 AM

May 3 2021

stallamr_netapp.com accepted D30094: ixgbe: Enable AIM by default.
May 3 2021, 8:15 PM

Feb 7 2021

stallamr_netapp.com added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

Review https://reviews.freebsd.org/D27465 helps to improve latency overall and has no dependency on this revision.

Feb 7 2021, 12:04 AM

Jan 27 2021

stallamr_netapp.com added inline comments to D27634: Ensure to free memory and pci resources in order.
Jan 27 2021, 9:50 PM
stallamr_netapp.com updated the diff for D27634: Ensure to free memory and pci resources in order.
  • Ensure to release MSIX after freeing IRQ resources
Jan 27 2021, 9:50 PM

Dec 22 2020

stallamr_netapp.com added inline comments to D27634: Ensure to free memory and pci resources in order.
Dec 22 2020, 6:36 PM

Dec 21 2020

stallamr_netapp.com updated the diff for D27634: Ensure to free memory and pci resources in order.
  • Add IFDI_QUEUES_FREE() to iflib_pseudo_deregister()
Dec 21 2020, 9:08 PM
stallamr_netapp.com added inline comments to D27634: Ensure to free memory and pci resources in order.
Dec 21 2020, 7:56 PM

Dec 16 2020

stallamr_netapp.com added reviewers for D27634: Ensure to free memory and pci resources in order: NetApp, krzysztof.galazka_intel.com.
Dec 16 2020, 5:17 PM
stallamr_netapp.com requested review of D27634: Ensure to free memory and pci resources in order.
Dec 16 2020, 5:15 PM

Dec 9 2020

stallamr_netapp.com added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

Addressed Mark J comments.

Dec 9 2020, 2:38 AM
stallamr_netapp.com updated the diff for D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..
  • Move AIM functionality into its own function to keep interrupt handler code simple
Dec 9 2020, 2:36 AM

Dec 3 2020

stallamr_netapp.com added inline comments to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..
Dec 3 2020, 8:41 PM
stallamr_netapp.com accepted D27465: iflib: Add rx_wait_irq configuration option.

This change with re-arm IRQ i.e, rx_wait_irq = 1, also helped NetApp to improve latency in 64K writes.

Dec 3 2020, 8:14 PM

Nov 24 2020

stallamr_netapp.com added a reviewer for D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration.: Intel Networking.
Nov 24 2020, 11:02 PM
stallamr_netapp.com updated the diff for D27342: Detach Rx/Tx/IOV tasks upon iflib device or psuedo device register failure..

rx bitmap do get free in following function call. Remove explicit free of bitmap yet again.

Nov 24 2020, 7:44 PM
stallamr_netapp.com added inline comments to D27342: Detach Rx/Tx/IOV tasks upon iflib device or psuedo device register failure..
Nov 24 2020, 6:35 PM
stallamr_netapp.com added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

Sure, thanks Kevin. I will discuss with Intel engineers.

Nov 24 2020, 12:00 AM

Nov 23 2020

stallamr_netapp.com added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

At NetApp, in one of our proprietary application 64K sequential write load test, we noticed that average LRO size of the packet/segment gets shorter. And we also observed, average LRO segment size is proportional to interrupt rate and throttling. For better running of our application, we need average LRO segment size to be on higher size. With AIM enabled (and with other Intel patch), the average LRO size now come close to BSD11 LRO average segment size for same application.

Nov 23 2020, 11:07 PM
stallamr_netapp.com added reviewers for D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration.: krzysztof.galazka_intel.com, jacob.e.keller_intel.com, NetApp.
Nov 23 2020, 8:00 PM
stallamr_netapp.com requested review of D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..
Nov 23 2020, 7:58 PM
stallamr_netapp.com added a reviewer for D27342: Detach Rx/Tx/IOV tasks upon iflib device or psuedo device register failure.: NetApp.
Nov 23 2020, 5:37 PM
stallamr_netapp.com requested review of D27342: Detach Rx/Tx/IOV tasks upon iflib device or psuedo device register failure..
Nov 23 2020, 5:36 PM

Nov 11 2020

stallamr_netapp.com requested review of D27179: Free full mbuf chain when draining Tx.
Nov 11 2020, 3:17 PM