Page MenuHomeFreeBSD

gallatin (Andrew Gallatin)
User

Projects

User Details

User Since
Jun 22 2015, 5:21 PM (190 w, 4 d)

Recent Activity

Yesterday

gallatin accepted D19201: iflib: Return the correct filter result in the interrupt handler..
Fri, Feb 15, 12:52 PM

Thu, Feb 14

gallatin added a comment to D19201: iflib: Return the correct filter result in the interrupt handler..

I understand the missing FILTER_STRAY.

Thu, Feb 14, 8:21 PM

Fri, Feb 8

gallatin updated the diff for D19063: mlx5en: add pfil ethernet hook.

Updated to truncate the length passed to pfil for multi-segment jumbo frames to just the length of the first segment.

Fri, Feb 8, 3:13 PM

Thu, Feb 7

gallatin added a comment to D19063: mlx5en: add pfil ethernet hook.

Any high performance interface will look similar to this and be similarly invasive.

Thu, Feb 7, 9:27 PM
gallatin added a comment to D19063: mlx5en: add pfil ethernet hook.

The reason to have this in the driver is for performance. The plan for pfil is to eventually just use a pointer to a memory blob for filtering. The current fake mbuf stuff is a step in that direction.

Thu, Feb 7, 6:35 PM

Wed, Feb 6

gallatin added inline comments to D19063: mlx5en: add pfil ethernet hook.
Wed, Feb 6, 3:58 PM
gallatin updated the diff for D19063: mlx5en: add pfil ethernet hook.
  • moved rv declaration to top of function
  • entered CURVNET to prevent panic from null curvnet when VIMAGE is compiled in Note placing this at the top of the rx function is a trade off between tiny overhead in the common case (nothing hooked) and a bit larger overhead in the case when something is hooked, and entering/restoring the VNET each time we call into the filter.
  • Addressed all possible return values from pfil_run_hooks Note that PFIL_REALLOCED was tested by hacking pfil.c to copy packets
  • Added a label to jump to when PFIL_REALLOCED is returned, and we need to send up an mbuf allocated by the filter.
Wed, Feb 6, 3:56 PM

Sat, Feb 2

gallatin created D19063: mlx5en: add pfil ethernet hook.
Sat, Feb 2, 3:29 PM

Thu, Jan 31

gallatin accepted D19040: Don't set IFCAP_TXRTLMT during lagg_clone_create()..
Thu, Jan 31, 12:15 AM

Mon, Jan 28

gallatin committed rS343526: MFC r343430.
MFC r343430
Mon, Jan 28, 2:35 PM

Fri, Jan 25

gallatin committed rS343430: Fix an iflib driver unload panic introduced in r343085.
Fix an iflib driver unload panic introduced in r343085
Fri, Jan 25, 3:02 PM

Tue, Jan 22

gallatin committed rS343304: MFC r341095:.
MFC r341095:
Tue, Jan 22, 5:35 PM

Jan 4 2019

gallatin committed rS342774: Limit git history searches in newvers.sh.
Limit git history searches in newvers.sh
Jan 4 2019, 6:38 PM
gallatin closed D18745: Limit git history searches in newvers.sh.
Jan 4 2019, 6:38 PM
gallatin updated the diff for D18745: Limit git history searches in newvers.sh.

Remove XXX as per wlosh

Jan 4 2019, 5:02 PM
gallatin updated the diff for D18745: Limit git history searches in newvers.sh.
Jan 4 2019, 4:25 PM
gallatin created D18745: Limit git history searches in newvers.sh.
Jan 4 2019, 4:23 PM

Jan 1 2019

gallatin accepted D18690: Reduce overhead of per-packet processing by ipfw(4).
Jan 1 2019, 9:10 PM

Dec 30 2018

gallatin accepted D18690: Reduce overhead of per-packet processing by ipfw(4).

I really like collapsing all those pointers, and making the NULL checks flags. Great work.

Dec 30 2018, 9:10 PM

Dec 15 2018

gallatin added a comment to D18532: Use byte-counting rather than packet counting for TX batch size.

I like this much better; hopefully the performance tests will show that it is an improvement.

Dec 15 2018, 1:39 PM

Dec 14 2018

gallatin added inline comments to D18532: Use byte-counting rather than packet counting for TX batch size.
Dec 14 2018, 1:26 AM

Nov 27 2018

gallatin closed D17901: Use busdma unconditionally in iflib.
Nov 27 2018, 8:02 PM
gallatin committed rS341095: Use busdma unconditionally in iflib.
Use busdma unconditionally in iflib
Nov 27 2018, 8:02 PM

Nov 16 2018

gallatin accepted D17998: Drop sbsndptr KPI.

I was about to say that cxgbe still uses it, but it looks like that was just changed in r340465. So yes, lets get rid of it.

Nov 16 2018, 2:05 PM

Nov 10 2018

gallatin updated the diff for D17901: Use busdma unconditionally in iflib.
  • Fixed a pre-existing bug where, when receiving small packets, rx clusters were not unmapped, yet they were re-mapped when refilling the ring. This was fixed by keeping track of the bus address in ifsd_ba, and simply re-using them rather than re-doing the virt to bus mapping
Nov 10 2018, 2:27 PM

Nov 9 2018

gallatin added a comment to D17901: Use busdma unconditionally in iflib.

Thanks Olivier. 5% is about what I had expected. I think I see a way to improve that though. We seem to be doing repeated virt to phys translation on clusters where we have copied out a small mbuf on the rx side in iflib_rxd_pkt_get(). We call rxd_frag_to_sd() with a FALSE arg, to prevent unmapping. Yet in _iflib_fl_refill(), we seem to always do the mapping, even when we do not re-allocate clusters. This is one bit that's going to be more expensive on low-end boxes, since it will now result in a busdma callback function being called. I think this is an actual bug that would impact systems w/IOMMUs

Nov 9 2018, 3:09 PM

Nov 8 2018

gallatin created D17901: Use busdma unconditionally in iflib.
Nov 8 2018, 12:06 AM

Oct 26 2018

gallatin accepted D17704: Handle an empty NUMA domain 0..
Oct 26 2018, 1:41 PM

Oct 23 2018

gallatin added a comment to D17665: Ensure mbuf send tags don't trigger on ifnets.
In D17665#377526, @np wrote:
Oct 23 2018, 8:26 PM

Oct 19 2018

gallatin accepted D17416: Initialize global domainsets during boot..
Oct 19 2018, 12:37 PM
gallatin added inline comments to D17418: Make malloc_domain(9) fall back to other domains..
Oct 19 2018, 12:30 PM

Oct 11 2018

gallatin added a comment to D17419: Use malloc_domainset(9) instead of malloc_domain(M_WAITOK)..

Stepping back for a second, are we going to get to a point where we can make malloc() and uma_zalloc() use the current thread's domainset policy, rather than one derived from the kernel object?

Oct 11 2018, 1:21 PM
gallatin added inline comments to D17416: Initialize global domainsets during boot..
Oct 11 2018, 1:09 PM

Oct 10 2018

gallatin accepted D17503: Run epoch calls sooner and more reliably.

FWIW, i have patched both of these changes (this + D17492) into the upstream ck git repo, and run them through the CK regression tests for epoch in userpace on my 32-core/64-thread amd, and they passed.

Oct 10 2018, 7:16 PM
gallatin accepted D17492: Fix early epoch calls.
Oct 10 2018, 7:15 PM

Oct 8 2018

gallatin accepted D17439: Make it possible to disable NUMA support with a tunable..
Oct 8 2018, 8:37 PM
gallatin accepted D17416: Initialize global domainsets during boot..
Oct 8 2018, 8:36 PM
gallatin accepted D17474: A couple of style fixes in recent TCP changes..
Oct 8 2018, 7:34 PM
gallatin added a comment to D17416: Initialize global domainsets during boot..

You mention kstacks. I have not gone through the rest of your reviews, but I was wondering if you were working on affinitizing kstacks. Did you plan to break the cache out by domain, or drop it for affinitized kstack allocations? The default is so undersized these days, we're mostly not getting the benefit of the cache anyway (and the cache might be pessimal, since it is protected by a global lock).

I wasn't working on affinity. Rather, I want to ensure that the kstack allocation policy allows allocations from any domain, ensuring that we can permit kstack swapins so long as at least one domain has some free pages. See D17304; it's really just a bug fix for 12.0.

I haven't thought much about the problems you listed. Do you have a proposal?

Oct 8 2018, 5:59 PM
gallatin added inline comments to D17416: Initialize global domainsets during boot..
Oct 8 2018, 5:55 PM
gallatin added inline comments to D17417: Refactor domainset iterators in preparation for use by malloc() and UMA..
Oct 8 2018, 2:44 PM
gallatin accepted D17416: Initialize global domainsets during boot..

You mention kstacks. I have not gone through the rest of your reviews, but I was wondering if you were working on affinitizing kstacks. Did you plan to break the cache out by domain, or drop it for affinitized kstack allocations? The default is so undersized these days, we're mostly not getting the benefit of the cache anyway (and the cache might be pessimal, since it is protected by a global lock).

Oct 8 2018, 12:58 PM

Oct 6 2018

gallatin accepted D17450: Fix three epoch leaks.
Oct 6 2018, 8:11 PM

Oct 5 2018

gallatin added a comment to D17418: Make malloc_domain(9) fall back to other domains..

Do we really need a new malloc variant? Couldn't this be done with a new malloc flag that was specific to malloc_domain(). Something like M_ALTOK or something? (or maybe M_EXACT if you want the default to be the other way around, and have malloc_domain() fall back by default)

Oct 5 2018, 4:51 PM

Oct 1 2018

gallatin closed D16836: Allow empty NUMA domains to support AMD 2990WX.

This was committed as r339043. Due to a cut and paste error, the wrong review was linked from the commit message, so the review was not auto-closed.

Oct 1 2018, 3:09 PM
gallatin committed rS339043: Allow empty NUMA memory domains to support Threadripper2.
Allow empty NUMA memory domains to support Threadripper2
Oct 1 2018, 2:14 PM

Sep 24 2018

gallatin added inline comments to D16836: Allow empty NUMA domains to support AMD 2990WX.
Sep 24 2018, 9:20 PM
gallatin added a comment to D16836: Allow empty NUMA domains to support AMD 2990WX.

kmem_back() now also needs updating.

Sep 24 2018, 1:12 PM
gallatin updated the diff for D16836: Allow empty NUMA domains to support AMD 2990WX.
  • Fixed spelling errors pointed out by alc
  • Updated to account for recent VM changes:
    • added empty domain guard in kmm_back()
    • fixed conflict in keg_fetch_slab() after r338755
Sep 24 2018, 1:11 PM

Sep 19 2018

gallatin accepted D17242: Add IFCAP_TSO6 for igb.
Sep 19 2018, 6:05 PM

Sep 17 2018

gallatin updated the diff for D16836: Allow empty NUMA domains to support AMD 2990WX.

Added check for empty domains in uma_large_malloc_domain(), and removed added blank line, as pointed out by Mark

Sep 17 2018, 4:55 PM

Sep 15 2018

gallatin accepted D17158: Fix capabilities handling for iflib drivers.
Sep 15 2018, 3:32 PM

Sep 13 2018

gallatin added a comment to D16836: Allow empty NUMA domains to support AMD 2990WX.

We already have code to renumber NUMA domains to avoid holes in the domain ID space (renumber_domains() in srat.c). Wouldn't it be simpler to apply a similar approach to empty domains, or is there some reason that won't work?

Sep 13 2018, 1:34 PM

Sep 10 2018

gallatin accepted D17109: Style fixes for in6_pcblookup_lbgroup()..
Sep 10 2018, 6:41 PM
gallatin added a comment to D17031: Fix synchronization of lbgroup access..

FWIW, as I mentioned in private email, the testing went fine. I verified that there were no apparent mem leaks, and that in_pcblbgroup_free_deferred() was called when I bounced nginx several times on a heavily loaded (> 90Gb/s) box.

Sep 10 2018, 6:40 PM

Sep 6 2018

gallatin accepted D17059: Enable options NUMA on amd64 GENERIC/MINIMAL:.

We've had it in our config at Netflix for so long that I forgot that it was not in GENERIC..

Sep 6 2018, 1:00 PM · NUMA

Aug 29 2018

gallatin accepted D16882: Handle case where mbuf == &txq when IFC_QFLUSH is set.

For what it is, Van's birthday might be a better choice. Eg, some non-valid pointer that would cause free or mfree to panic rather than accept it and corrupt data.

Aug 29 2018, 10:08 AM

Aug 28 2018

gallatin added a comment to D16882: Handle case where mbuf == &txq when IFC_QFLUSH is set.

Sorry, having the ring pointer in the ring is just so odd.. Can you explain why it is there? Maybe it would be better to just not have the ring pointer in the ring and not have to have all these special cases.

Aug 28 2018, 10:58 PM
gallatin added a comment to D16882: Handle case where mbuf == &txq when IFC_QFLUSH is set.

What keeps the normal, non-IFC_QFLUSH from hitting this condition? Is the txq pointer included in r->size?

Aug 28 2018, 6:20 PM

Aug 27 2018

gallatin committed rS338341: Reject IPv4 SO_REUSEPORT_LB groups when looking up an IPv6 listening socket.
Reject IPv4 SO_REUSEPORT_LB groups when looking up an IPv6 listening socket
Aug 27 2018, 6:13 PM
gallatin closed D16899: SO_REUSEPORT_LB critical bug fix: avoid IPv4 listen group in IPv6.
Aug 27 2018, 6:13 PM

Aug 25 2018

gallatin added inline comments to D16855: Fix temperature reporting on AMD 2990WX.
Aug 25 2018, 11:10 PM
gallatin updated the diff for D16855: Fix temperature reporting on AMD 2990WX.
Aug 25 2018, 11:09 PM
gallatin created D16899: SO_REUSEPORT_LB critical bug fix: avoid IPv4 listen group in IPv6.
Aug 25 2018, 2:43 PM

Aug 23 2018

gallatin accepted D16875: Possible fix for "desc avail = " dmesg spam and igb breakage.
Aug 23 2018, 11:57 PM
gallatin added a comment to D16836: Allow empty NUMA domains to support AMD 2990WX.

I haven't had much of a chance to review this yet, but I wanted to test an "options NUMA" kernel on a system with an empty domain (dual socket Haswell xeon with only one DIMM installed at the moment). This patch worked fine for me. Thanks!

Aug 23 2018, 9:37 PM
gallatin added inline comments to D16836: Allow empty NUMA domains to support AMD 2990WX.
Aug 23 2018, 6:25 PM
gallatin updated the diff for D16836: Allow empty NUMA domains to support AMD 2990WX.
Aug 23 2018, 6:23 PM
gallatin added a comment to D16855: Fix temperature reporting on AMD 2990WX.

Thanks for the clarification on the non-offset range!

Aug 23 2018, 6:19 PM
gallatin updated the diff for D16855: Fix temperature reporting on AMD 2990WX.
Aug 23 2018, 6:18 PM
gallatin updated the diff for D16855: Fix temperature reporting on AMD 2990WX.
Aug 23 2018, 2:29 PM
gallatin added a comment to D16855: Fix temperature reporting on AMD 2990WX.

Thank you so much for the pointer to the docs!!! It is much nicer to write from real docs, rather than crib from undocumented changes in another project.

Aug 23 2018, 2:22 PM
gallatin accepted D16733: iflib sysctl cleanup.
Aug 23 2018, 12:46 PM
gallatin created D16855: Fix temperature reporting on AMD 2990WX.
Aug 23 2018, 12:06 AM

Aug 22 2018

gallatin added inline comments to D16836: Allow empty NUMA domains to support AMD 2990WX.
Aug 22 2018, 5:45 PM

Aug 21 2018

gallatin created D16836: Allow empty NUMA domains to support AMD 2990WX.
Aug 21 2018, 8:39 PM

Aug 17 2018

gallatin added inline comments to D16429: ixlv(4): Update to use iflib; change name to iavf(4).
Aug 17 2018, 7:35 PM · Intel Networking

Aug 16 2018

gallatin added inline comments to D16733: iflib sysctl cleanup.
Aug 16 2018, 1:03 PM

Aug 15 2018

gallatin added a reviewer for D16731: if_media: Add new 2.5G/5G/25G/40G/50G/100G/200G/400G media types: np.
Aug 15 2018, 7:03 PM

Aug 13 2018

gallatin committed rS337709: lagg: allow lacp to manage the link state.
lagg: allow lacp to manage the link state
Aug 13 2018, 2:13 PM

Aug 10 2018

gallatin accepted D16429: ixlv(4): Update to use iflib; change name to iavf(4).
Aug 10 2018, 3:41 PM · Intel Networking

Jul 30 2018

gallatin added a comment to D16429: ixlv(4): Update to use iflib; change name to iavf(4).

In general, I looked around at ixlv.c, and it looks like there are many functions for the main pf which are identical, or nearly identical for the VF. Can we please work on collapsing them down to common code?

Jul 30 2018, 4:32 PM · Intel Networking

Jul 17 2018

gallatin accepted D16302: Add knob to control tx ring abdication..

Thank for doing this!

Jul 17 2018, 7:10 PM

Jul 5 2018

gallatin committed rS335973: mxge: replace 65536 with IP_MAXPACKET in tso settings..
mxge: replace 65536 with IP_MAXPACKET in tso settings.
Jul 5 2018, 2:43 AM

Jul 4 2018

gallatin committed rS335967: mxge: choose appropriate values for hw tso.
mxge: choose appropriate values for hw tso
Jul 4 2018, 7:29 PM
gallatin committed rS335966: mxge: Add SIOCGI2C support for devices with SFP/XFP cages.
mxge: Add SIOCGI2C support for devices with SFP/XFP cages
Jul 4 2018, 6:54 PM
gallatin committed rS335957: mxge: fix panic at module unload.
mxge: fix panic at module unload
Jul 4 2018, 2:26 PM

Jun 27 2018

gallatin added a comment to D15985: Reduce unnecessary preemption, add a preemption knob for timeshare, fix missing NEEDRESCHED.

I ran this on a Netflix 100g box. I observed no measurable difference in CPU time. So I think this patch is "neutral" for the perspective of our (mostly kernel) workload.

Jun 27 2018, 1:32 PM

Jun 20 2018

gallatin added a comment to D15937: Optimize the TSO and copy paths to use the new tcp_m_copy routine.

I've deployed a variant of this widely at LLNW on the default stack as well. I think you can garbage collect sbsndptr() from the tree.

Jun 20 2018, 9:30 PM
gallatin accepted D15937: Optimize the TSO and copy paths to use the new tcp_m_copy routine.

I added Hans since it is mostly his code we are factoring out.

Jun 20 2018, 8:40 PM
gallatin added a reviewer for D15937: Optimize the TSO and copy paths to use the new tcp_m_copy routine: hselasky.
Jun 20 2018, 8:39 PM

Jun 14 2018

gallatin added a comment to D15686: convert inpcbhash rlock to epoch.

if @jch cannot run his real workload, what about running a synthetic test that creates / tears down many connections per second? Eg, something like many copies of netperf -t TCP_CC ?

Jun 14 2018, 4:23 PM
gallatin accepted D15577: Update ixl(4) to use iflib..
Jun 14 2018, 2:53 AM
gallatin added a comment to D15577: Update ixl(4) to use iflib..

You might want to go for some middle ground here. 1ms is a bit much (even for us).

Jun 14 2018, 2:52 AM

Jun 11 2018

gallatin added inline comments to D15577: Update ixl(4) to use iflib..
Jun 11 2018, 11:41 PM

Jun 4 2018

gallatin accepted D15558: iflib: Record TCP checksum info in iflib for ixl(4).

Sure.. Those checks for TCP in the TSO case should probably turn into asserts eventually..

Jun 4 2018, 8:13 PM · Intel Networking

May 30 2018

gallatin accepted D15526: reduce overhead of entropy collection.

If people want to use ethernet entropy harvesting, can we do something to make it more useful in a separate change? I was going to initially suggest the ether dst addr, but that's not very random either.

May 30 2018, 9:22 PM
gallatin added a reviewer for D15526: reduce overhead of entropy collection: gallatin.
May 30 2018, 9:18 PM

May 29 2018

gallatin added a comment to D15577: Update ixl(4) to use iflib..

This is the sparse detect I'm using. Note how it walks the header, and then the TSO by packet on the wire.

May 29 2018, 9:28 PM
gallatin added a comment to D15577: Update ixl(4) to use iflib..

Somehow my last comment got lost. With real Netflix traffic, the sparse detection logic is not sufficient. When we have kernel TLS, which inserts 13 and 16 byte mbufs around TLS records, we hit MDDs regularly. I think you need to walk the packet stream, not the DMA descriptors, so that you keep the same segmentation used by the hardware. I re-wrote the sparse detection to do this, and our MDDs went away.

May 29 2018, 9:27 PM