Page MenuHomeFreeBSD

gallatin (Andrew Gallatin)
User

Projects

User Details

User Since
Jun 22 2015, 5:21 PM (258 w, 1 d)

Recent Activity

Wed, May 27

gallatin accepted D24674: fix sosend() so that it can handle an mbuf list for KTLS.
Wed, May 27, 3:48 AM

Tue, May 26

gallatin added a reviewer for D25018: cpuset: specify that we're printing a list of cpus, not a mask: jeff.
Tue, May 26, 3:41 PM

Wed, May 20

gallatin accepted D24922: bnxt: isc_nrxd_max and isc_ntxd_max must be powers of two.
Wed, May 20, 3:23 PM
gallatin accepted D24922: bnxt: isc_nrxd_max and isc_ntxd_max must be powers of two.
Wed, May 20, 1:55 PM
gallatin added a reviewer for D24922: bnxt: isc_nrxd_max and isc_ntxd_max must be powers of two: gallatin.
Wed, May 20, 1:54 PM

Tue, May 12

gallatin closed D24821: IPv6: Fix a panic in the nd6 code with unmapped mbufs..
Tue, May 12, 5:18 PM
gallatin committed rS360982: IPv6: Fix a panic in the nd6 code with unmapped mbufs..
IPv6: Fix a panic in the nd6 code with unmapped mbufs.
Tue, May 12, 5:18 PM
gallatin closed D24822: IPv6: sync IP_NO_SND_TAG_RL support from IPv4.
Tue, May 12, 2:01 PM
gallatin committed rS360961: IPv6: sync IP_NO_SND_TAG_RL support from IPv4.
IPv6: sync IP_NO_SND_TAG_RL support from IPv4
Tue, May 12, 2:01 PM
gallatin requested review of D24822: IPv6: sync IP_NO_SND_TAG_RL support from IPv4.
Tue, May 12, 1:48 PM
gallatin requested review of D24821: IPv6: Fix a panic in the nd6 code with unmapped mbufs..
Tue, May 12, 1:35 PM

Mon, May 11

gallatin committed rS360930: Fix the build.
Fix the build
Mon, May 11, 9:23 PM
gallatin committed rS360914: Ktls: never skip stamping tags for NIC TLS.
Ktls: never skip stamping tags for NIC TLS
Mon, May 11, 7:17 PM

Thu, May 7

gallatin accepted D24659: em/ix/ixv/ixl/iavf: Implement ifdi_needs_restart iflib method.
Thu, May 7, 3:21 AM

Wed, May 6

gallatin accepted D24736: Remove un-needed NF copy args.
Wed, May 6, 9:26 PM

May 1 2020

gallatin accepted D24659: em/ix/ixv/ixl/iavf: Implement ifdi_needs_restart iflib method.
May 1 2020, 11:07 PM
gallatin added a comment to D24645: sched_ule: rate limit work stealing.
In D24645#542519, @mav wrote:

I was thinking about something like this many times. It would be nice to skip this often useless work. But each time I am stopped by worry that it may cause CPU under-utilization on a systems with load average about 1 per SMT thread. 10ms interval you mentioned sounds quite large to me, but honestly I can imagine some synchronous irregular load patterns where any chosen interval will cause a pessimization, dramatically increasing execution latency.

May 1 2020, 1:42 PM

Apr 30 2020

gallatin requested review of D24645: sched_ule: rate limit work stealing.
Apr 30 2020, 11:38 PM

Apr 28 2020

gallatin added inline comments to D24598: More re-working of multipage mbufs..
Apr 28 2020, 1:39 PM

Apr 27 2020

gallatin accepted D24598: More re-working of multipage mbufs..
Apr 27 2020, 11:49 PM
gallatin accepted D24568: fix sosend_generic() so that it can handle a list of ext_pgs mbufs post r359919.
Apr 27 2020, 1:57 PM
gallatin accepted D24545: Add support for optional separate output buffers to in-kernel crypto..

Looked mostly at ktls

Apr 27 2020, 1:56 PM
gallatin added a comment to D24453: Add support for KTLS RX over TOE to T6..

I'm going to let Navdeep review this..

Apr 27 2020, 1:18 PM
gallatin accepted D24452: Initial support for kernel offload of TLS receive..
Apr 27 2020, 1:17 PM
gallatin accepted D24451: Add the initial sequence number to the TLS enable socket option..
Apr 27 2020, 1:12 PM

Apr 14 2020

gallatin committed rS359920: Bump FreeBSD version after r359919 (KTLS / unmapped mbuf changes).
Bump FreeBSD version after r359919 (KTLS / unmapped mbuf changes)
Apr 14 2020, 2:48 PM
gallatin closed D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
Apr 14 2020, 2:46 PM
gallatin committed rS359919: KTLS: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
KTLS: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself.
Apr 14 2020, 2:46 PM

Apr 13 2020

gallatin committed rS359908: lagg: stop double-counting output errors and counting drops as errors.
lagg: stop double-counting output errors and counting drops as errors
Apr 13 2020, 11:07 PM
gallatin closed D24331: lagg: stop double-counting output errors and counting drops as errors.
Apr 13 2020, 11:07 PM
gallatin added a comment to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..

Just a heads up that i plan to commit this tomorrow.

Apr 13 2020, 2:22 PM

Apr 9 2020

gallatin updated the diff for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..

Fix issues encountered when running mlx5 with hw tls enabled with this patchset:

    • accessing ext_pgs via pointer was missed at compile time, as the driver was using the ext_buf rather than ext_pgs pointer
  • the driver was using ext_pgs at the same time as pkthdr
Apr 9 2020, 5:37 PM
gallatin updated the diff for D24331: lagg: stop double-counting output errors and counting drops as errors.
  • Fix accounting issue that jhb pointed out in lagg_bcast_start()
Apr 9 2020, 1:33 PM

Apr 8 2020

gallatin added inline comments to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
Apr 8 2020, 3:11 PM
gallatin updated the diff for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..

Make cosmetic/style fixes as suggested by jhb

Apr 8 2020, 3:11 PM
gallatin updated the diff for D24331: lagg: stop double-counting output errors and counting drops as errors.

Update to reflect jhb's feedback. We can stop checking the ret value of lagg_enqueue() inside the loop entirely, as it will be overwritten the next time it is used outside the loop, so this simplifies things as well.

Apr 8 2020, 2:16 PM
gallatin accepted D22086: iflib: Stop interface before (un)registering VLAN.
Apr 8 2020, 1:53 PM

Apr 7 2020

gallatin created D24331: lagg: stop double-counting output errors and counting drops as errors.
Apr 7 2020, 8:32 PM

Apr 6 2020

gallatin updated the diff for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
  • Rebase past r359474
  • Move m_ext_pgs down below m_pkthdr in the union, as suggested by Hans
Apr 6 2020, 6:30 PM
gallatin added a reviewer for D24274: security/openssl: Add support for in-kernel TLS (KTLS).: hselasky.
Apr 6 2020, 2:47 PM
gallatin accepted D24274: security/openssl: Add support for in-kernel TLS (KTLS)..
Apr 6 2020, 2:26 PM

Apr 3 2020

gallatin created D24273: mlx5en: increment enobuf counter when tx schedule is out of space.
Apr 3 2020, 8:51 PM

Apr 1 2020

gallatin added inline comments to D24236: Fix panic in iflib when out of mbuf clusters.
Apr 1 2020, 8:22 PM
gallatin added a comment to D24236: Fix panic in iflib when out of mbuf clusters.

Picture a case where you're asked to refill 128 slots, and fail after 127. Don't you want to at least flush the 127 that you were able to allocate?

Apr 1 2020, 4:58 PM

Mar 30 2020

gallatin closed D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.
Mar 30 2020, 11:30 PM
gallatin committed rS359474: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.
KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency
Mar 30 2020, 11:30 PM
gallatin updated the diff for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.

Comment changes suggested by jhb

Mar 30 2020, 6:34 PM
gallatin added a comment to D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.

Change looks good.

One comment:
Style is bcopy() and not memcpy() ?

Mar 30 2020, 3:44 PM
gallatin updated the diff for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.
  • switch from bcopy to memcpy as suggested by Hans
Mar 30 2020, 3:43 PM
gallatin updated the diff for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.
  • Remove ifdef KERN_TLS from locals' declarations and move those into the ifdef'ed block as suggested by scottl
  • Add comment regarding where n comes from as suggested by scottl
Mar 30 2020, 3:41 PM
gallatin updated the diff for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..

After discussion with Hans, it turns out that it is the record type, not the seqno, that the Mellanox driver needs in the trailer for hwtls.

Mar 30 2020, 2:53 PM
gallatin added inline comments to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
Mar 30 2020, 1:40 PM
gallatin added a comment to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
In D24213#532734, @kib wrote:
In D24213#532679, @kib wrote:

Don't you want to provide some asserts about the struct mbuf layout ?
Imagine I do not know anything about constraints and want to add a field somewhere.

In kern_mbuf.c, I replaced the constraint that m_ext_pgs was 256b with the constraints that the offset of m_ext in struct mbuf matches the offset of m_ext_pgs.m_ext, as well as a constraint that the size of struct mbuf does not exceed MSIZE. Did you have others in mind? The ones I added caught a few problems on 32-bit platforms which naturally align 8-byte types, for example.

No, I do not, otherwise I would suggest the checks.

BTW, you commented out some asserts, why not remove them if they are not relevant ?

Mar 30 2020, 1:38 PM
gallatin added inline comments to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
Mar 30 2020, 1:37 PM
gallatin updated the diff for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
  • Restored static asserts on the size of m_ext() that I'd forgotten I'd commented
  • Changed copy of seqno to a memcpy
Mar 30 2020, 1:33 PM

Mar 29 2020

gallatin added a comment to D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
In D24213#532679, @kib wrote:

Don't you want to provide some asserts about the struct mbuf layout ?
Imagine I do not know anything about constraints and want to add a field somewhere.

Mar 29 2020, 1:34 PM

Mar 28 2020

gallatin added a reviewer for D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself.: glebius.
Mar 28 2020, 8:28 PM
gallatin created D24213: Re-work unmapped mbufs to carry ext_pgs in the mbuf itself..
Mar 28 2020, 8:27 PM

Mar 27 2020

gallatin added a reviewer for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency: scottl.
Mar 27 2020, 7:45 PM
gallatin added a reviewer for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency: np.
Mar 27 2020, 3:15 PM
gallatin added reviewers for D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency: jhb, hselasky, rrs, jtl.
Mar 27 2020, 3:15 PM
gallatin created D24204: KTLS: Coalesce adjacent TLS trailers & headers to improve PCIe bus efficiency.
Mar 27 2020, 3:14 PM

Mar 16 2020

gallatin committed rS359016: Avoid a cache miss accessing an mbuf ext_pgs pointer when doing SW kTLS..
Avoid a cache miss accessing an mbuf ext_pgs pointer when doing SW kTLS.
Mar 16 2020, 2:03 PM
gallatin closed D23998: Don't deref an mbuf ext_pgs pointer in tcp_m_copym() unless we need to.
Mar 16 2020, 2:03 PM

Mar 9 2020

gallatin closed D23811: make lacp's use_numa hashing aware of send tags.

Committed as r358808. (however I forgot to tag the review in the commit message)

Mar 9 2020, 1:47 PM
gallatin committed rS358808: make lacp's use_numa hashing aware of send tags.
make lacp's use_numa hashing aware of send tags
Mar 9 2020, 1:45 PM

Mar 7 2020

gallatin created D23998: Don't deref an mbuf ext_pgs pointer in tcp_m_copym() unless we need to.
Mar 7 2020, 10:57 PM

Mar 3 2020

gallatin added a comment to D23948: Remove refill budget from iflib.

I get it, but that (totally arbitrary) limit is a pet peeve of mine. Its one of those things that you follow up several levels of code to an "XXX" comment and an arbitrary value. Its like reading a mystery novel and finding the butler really did do it.

Mar 3 2020, 7:12 PM
gallatin accepted D23943: Fix iflib freelist state corruption.
Mar 3 2020, 6:25 PM
gallatin accepted D23946: Remove freelist contiguous-indexes assertion from rxd_frag_to_sd().

Funny how the comment right above the assert calls it out as bogus..

Mar 3 2020, 6:20 PM
gallatin accepted D23951: Remove extraneous code from iflib.

Thank you for this cleanup.

Mar 3 2020, 6:18 PM
gallatin accepted D23950: Adjust if_vmx default receive ring size and receive ring buffers sizes to improve out-of-box performance.
Mar 3 2020, 6:14 PM
gallatin accepted D23949: Fix if_vmx receive checksum offload bug and harden against the device skipping receive descriptors.
Mar 3 2020, 6:12 PM
gallatin added a comment to D23948: Remove refill budget from iflib.

Why not just remove the limit, rather than making things even more complex?

Mar 3 2020, 6:05 PM
gallatin accepted D23947: Allow iflib drivers to specify the buffer size used for each receive queue.
Mar 3 2020, 6:01 PM
gallatin accepted D23945: Fix iflib zero-length fragment handling.
Mar 3 2020, 5:31 PM

Feb 27 2020

gallatin accepted D23726: Make RSS kernels compile again.
Feb 27 2020, 3:20 PM

Feb 24 2020

gallatin accepted D23760: Fix IPv6 checksums when exthdrs are present..
Feb 24 2020, 6:28 PM

Feb 23 2020

gallatin created D23811: make lacp's use_numa hashing aware of send tags.
Feb 23 2020, 9:27 PM

Feb 21 2020

gallatin added inline comments to D23760: Fix IPv6 checksums when exthdrs are present..
Feb 21 2020, 12:50 AM

Feb 20 2020

gallatin added inline comments to D23760: Fix IPv6 checksums when exthdrs are present..
Feb 20 2020, 3:46 PM
gallatin abandoned D23752: Teach lagg to use rate-limited send tags for lacp.

Abandoning this in favor of moving selection of an uncongested queue for lacp into the mlx5 driver.

Feb 20 2020, 3:37 PM
gallatin added inline comments to D23760: Fix IPv6 checksums when exthdrs are present..
Feb 20 2020, 2:56 PM
gallatin accepted D23760: Fix IPv6 checksums when exthdrs are present..
Feb 20 2020, 1:48 PM

Feb 19 2020

gallatin added a comment to D23752: Teach lagg to use rate-limited send tags for lacp.
In D23752#521981, @np wrote:

This doesn't look right to me.

Rate limiting a connection doesn't have much to do with tx queue
selection except on mlx5_en but this patch adds this assumption to lagg.
Even in theory this can work only with kernels with RATELIMIT (GENERIC
doesn't have it) and then only with the two NICs that support RATELIMIT
(cxgbe and mlx5_en). In practice this might fix the problem for mlx5_en
but will make it worse for cxgbe.

Scott added rsrv_noflowq to cxgbe in r261558 for this very reason. That
is driver-specific too but at least the change is entirely within the
driver and doesn't pollute sys/net. Can't something like that be done here?

Feb 19 2020, 8:54 PM
gallatin added a comment to D23760: Fix IPv6 checksums when exthdrs are present..

I'm 1/2 joking, but what would you think about not supporting extension headers at all? They are the worst part of IPv6 and make everything complicated and add lots of hairy cases. What benefit are they?
(I'm legitimately curious)

Feb 19 2020, 8:51 PM
gallatin accepted D23740: ip6_output: improve extension header handling.
Feb 19 2020, 6:14 PM
gallatin updated the diff for D23752: Teach lagg to use rate-limited send tags for lacp.
Feb 19 2020, 2:34 AM
gallatin created D23752: Teach lagg to use rate-limited send tags for lacp.
Feb 19 2020, 2:30 AM

Feb 18 2020

gallatin accepted D23710: Fix panics when using iflib pseudo device support.

I'd prefer the check be moved inside _iflib_assert(), but that's just a nit.

Feb 18 2020, 2:31 PM
gallatin added inline comments to D23718: Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (M of N).
Feb 18 2020, 2:22 PM

Feb 13 2020

gallatin accepted D23674: Change smarter epoch support for NIC drivers to opt-in..
Feb 13 2020, 10:33 PM

Feb 5 2020

gallatin added a comment to D23483: Fix for unbalanced EPOCH(9) usage in kernel interrupt handler.

What we really care about is not re-ordering the load so that we ensure it happens prior to the loop. So I think you want an atomic with acquire semantics:

"the operation must have completed before any subsequent load or store (by program order) is performed"
Feb 5 2020, 3:24 PM

Feb 3 2020

gallatin accepted D23483: Fix for unbalanced EPOCH(9) usage in kernel interrupt handler.

To expand on this, the issue is that interrupt handlers are removed via intr_event_execute_handlers() when IH_DEAD is set. The thread removing the intr is woken up, and he calls intr_event_update(). When this happens, the ie_hflags are cleared and re-built from all the remaining handlers sharing the event. When the last IH_NET handler is removed, the IH_NET flag will be cleared from ih_hflags (or ie_hflags may still be being rebuilt in a different context), and the ithread_execute_handlers() may return with ie_hflags missing IH_NET. So we can end up in a scenario where IH_NET was present before calling ithread_execute_handlers, and is not present at its return, meaning we must cache the need for epoch locally.

Feb 3 2020, 4:03 PM

Feb 1 2020

gallatin added inline comments to D23408: epoch support for taskqueues.
Feb 1 2020, 3:43 PM

Jan 27 2020

gallatin accepted D23379: Enter epoch in msk_tick.
Jan 27 2020, 12:06 AM

Jan 24 2020

gallatin requested changes to D23348: Widen EPOCH(9) usage in network drivers (as a pre-step for D23347).

I like the idea of using a different ether_input() for epoch and non-epoch safe drivers to get the tests out of the critical path. However, this is a minor optimization at best, as branch predictors are really good these days on modern CPUs.

Jan 24 2020, 2:15 PM
gallatin requested changes to D23347: Add support for batching interrupts in system interrupt code.
Jan 24 2020, 2:11 PM
gallatin added a comment to D23347: Add support for batching interrupts in system interrupt code.

Some feedback:

Jan 24 2020, 2:11 PM

Jan 22 2020

gallatin added a comment to D23315: Make sure end of receive doesn't cause interrupt starvation in iflib.

As we discussed in slack, I'm not a huge fan of fixing things with a callout. If the hardware was amenable, I'd much rather leave the ring partially stocked and drop new packets until we were able to allocate mbufs again. However, based on the findings you reported with igb (it wanting a full rx ring to generate interrupts), I'm afraid that might require too much work from hardware drivers. And I'd prefer a fix to a real problem, even if its not something I personally find appealing, to leaving a real bug unfixed and having machines become unreachable.

Jan 22 2020, 10:18 PM