Page MenuHomeFreeBSD

gallatin (Andrew Gallatin)
User

Projects

User Details

User Since
Jun 22 2015, 5:21 PM (296 w, 3 d)

Recent Activity

Tue, Feb 23

gallatin accepted D28900: iflib: Avoid double counting in rxeof.
Tue, Feb 23, 8:16 PM

Mon, Feb 22

gallatin accepted D28706: iflib: add new field: isc_dma_width.
Mon, Feb 22, 1:23 AM

Fri, Feb 19

gallatin accepted D28808: m_uiotombuf_nomap(): Stop clearing PG_ZERO.
Fri, Feb 19, 10:04 PM
gallatin accepted D28724: Update manpage for AMD 10GbE driver.
Fri, Feb 19, 2:49 PM

Thu, Feb 18

gallatin added inline comments to D28724: Update manpage for AMD 10GbE driver.
Thu, Feb 18, 2:59 PM

Wed, Feb 17

gallatin accepted D27841: Add Chacha20-Poly1305 support in the OCF backend for KTLS..

Maybe add a comment for the else cases saying that they are chacha?

Wed, Feb 17, 11:54 PM
gallatin accepted D28374: Compressed ack enhancements to LRO.
Wed, Feb 17, 2:14 PM
gallatin added a comment to D28709: gicv3_its: Leave LPI interrupts enable during handling.

I've tested this on a Netflix server, and can confirm that it works fine, and eliminates spinlock contention on the gic cmd queue.

Wed, Feb 17, 1:21 AM

Tue, Feb 16

gallatin accepted D28587: Documentation repo changes for adding new committer Rajesh Kumar.
Tue, Feb 16, 9:15 PM
gallatin added a comment to D28714: Handle partial data re-sending on ktls/sendfile on FreeBSD.

I've tested this on Netflix servers, and it seems to work.

Tue, Feb 16, 8:23 PM
gallatin added a comment to D28709: gicv3_its: Leave LPI interrupts enable during handling.

Are legacy interrupts supported on this platform? Is this code also used for legacy interrupts?

Tue, Feb 16, 6:18 PM
gallatin accepted D28706: iflib: add new field: isc_dma_width.

I'm shocked that this feature was not present before.

Tue, Feb 16, 5:02 PM
gallatin added inline comments to D28556: ktls: Use large pages for output when encrypting in software.
Tue, Feb 16, 4:47 PM
gallatin added a comment to D28556: ktls: Use large pages for output when encrypting in software.

This is nothing short of amazing. It cuts CPU use on an original Netflix 100G server (16 core Broadwell Xeon 2697A v4) from ~62% to ~52% at roughly 92Gb/s of a 100% ktls Netflix video serving workload.

Tue, Feb 16, 1:12 AM

Fri, Feb 12

gallatin accepted D28587: Documentation repo changes for adding new committer Rajesh Kumar.
Fri, Feb 12, 3:44 AM

Thu, Feb 11

gallatin added inline comments to D28587: Documentation repo changes for adding new committer Rajesh Kumar.
Thu, Feb 11, 5:03 PM
gallatin accepted D28586: Source repo changes for adding new committer Rajesh Kumar.
Thu, Feb 11, 5:03 PM

Wed, Feb 10

gallatin added a comment to D28556: ktls: Use large pages for output when encrypting in software.

This is pretty exciting. I can try this on amd64 and let you know how much it helps.

Thanks! I expect the improvement to be modest but that'll help inform whether this is worth doing.

It won't be until next week, as I have a few things going on this week, and it looks like I may need to change the ktls_isa-l_crypto-kmod to catch up to having iniovcnt != outiovcnt

I guess that's not in the tree? That would hopefully be a straightforward conversion.

Wed, Feb 10, 3:12 PM

Tue, Feb 9

gallatin added a comment to D28556: ktls: Use large pages for output when encrypting in software.

This is pretty exciting. I can try this on amd64 and let you know how much it helps. It won't be until next week, as I have a few things going on this week, and it looks like I may need to change the ktls_isa-l_crypto-kmod to catch up to having iniovcnt != outiovcnt

Tue, Feb 9, 6:39 PM

Sun, Feb 7

gallatin added a comment to D28508: mlx5: Pad sq locks to minimize false sharing.

The producer and consumer certainly may be on different CPUs.

Sun, Feb 7, 6:49 PM

Fri, Feb 5

gallatin accepted D28499: ktls: Make configuration sysctls available as tunables.
Fri, Feb 5, 2:48 PM

Wed, Jan 27

gallatin accepted D28377: m_nextpkt pointer not NULL'ed before sending.

This probably needs to be MFC'ed to 13

Wed, Jan 27, 6:38 PM

Jan 22 2021

gallatin added a comment to D28273: OpenSSL: Support for kernel TLS offload (KTLS)..

Note that we (Netflix) have been running this patch in evolving state since 2017 in production.

Jan 22 2021, 1:04 AM

Jan 21 2021

gallatin closed D28115: Fix a NULL pointer reference in iflib.
Jan 21 2021, 2:51 PM
gallatin committed R10:0c864213ef1e: iflib: Fix a NULL pointer deref (authored by gallatin).
iflib: Fix a NULL pointer deref
Jan 21 2021, 2:50 PM

Jan 18 2021

gallatin closed D28163: KTLS: Enable KERN_TLS in GENERIC on amd64.
Jan 18 2021, 6:31 PM
gallatin committed R10:efa9c21bca98: KTLS: Enable KERN_TLS in GENERIC on amd64 (authored by gallatin).
KTLS: Enable KERN_TLS in GENERIC on amd64
Jan 18 2021, 6:31 PM

Jan 15 2021

gallatin added a comment to D27797: AMD 10GbE driver changes for Netmap support.

I see "iflib" group review is blocking here. Now that this patch is only driver specific and doesn't have any change in iflib, does this patch need any other action to get it pushed upstream?

Jan 15 2021, 5:12 PM

Jan 14 2021

gallatin accepted D28161: ktls: Improve handling of the bind_threads tunable a bit.
Jan 14 2021, 6:15 PM
gallatin accepted D28160: netipsec: Fix handling of unmapped mbufs.

Thank you for catching this.

Jan 14 2021, 6:14 PM
gallatin requested review of D28163: KTLS: Enable KERN_TLS in GENERIC on amd64.
Jan 14 2021, 5:58 PM

Jan 13 2021

gallatin accepted D28115: Fix a NULL pointer reference in iflib.

OK, thanks that makes sense. I was not considering a multi-frag case.

Jan 13 2021, 6:27 PM

Jan 12 2021

gallatin added a comment to D28115: Fix a NULL pointer reference in iflib.

Whats the path to having a null pointer as pf_rv ?

Jan 12 2021, 2:09 PM
gallatin committed R10:7eaea04a5bb1: amd64: compare TLB shootdown target to all_cpus (authored by gallatin).
amd64: compare TLB shootdown target to all_cpus
Jan 12 2021, 1:10 AM
gallatin closed D28102: amd64: compare TLB shootdown target to all_cpus.
Jan 12 2021, 1:10 AM
gallatin added a comment to D28102: amd64: compare TLB shootdown target to all_cpus.

i386 should probably get the same patch?

Jan 12 2021, 1:06 AM

Jan 11 2021

gallatin updated the summary of D28102: amd64: compare TLB shootdown target to all_cpus.
Jan 11 2021, 9:26 PM
gallatin added reviewers for D28102: amd64: compare TLB shootdown target to all_cpus: kib, jhb, markj, jtl.
Jan 11 2021, 9:24 PM
gallatin requested review of D28102: amd64: compare TLB shootdown target to all_cpus.
Jan 11 2021, 9:24 PM

Jan 8 2021

gallatin committed R10:52cd25eb1aa7: mbuf: enable ext_pgs ("unmapped") mbufs by default (authored by gallatin).
mbuf: enable ext_pgs ("unmapped") mbufs by default
Jan 8 2021, 6:44 PM

Jan 7 2021

gallatin added a comment to D27755: Add batched packet processing support..

There is already a mechanism drivers can use to build an array of packets which is used by LRO. Eg, tcp_lro_queue_mbuf(). It was quite helpful, not so much for the batching, but because it sorts the array to put packets from the same TCP connection adjacent to each other..

Jan 7 2021, 10:15 PM
gallatin accepted D27986: x86 tsc: use RDTSCP in preference of fence + RDTSC.
Jan 7 2021, 9:01 PM
gallatin added a comment to D27986: x86 tsc: use RDTSCP in preference of fence + RDTSC.

After talking with our AMD rep, the AMD advise for this is:

Jan 7 2021, 8:57 PM
gallatin added a comment to D27755: Add batched packet processing support..

Might it be better to just make a new if_input method which takes an array of mbufs? List processing for a batch of packets seems like a recipe for cache misses..

Jan 7 2021, 7:50 PM
gallatin committed R10:a2fc8ade1057: isci: use maxphys rather than 128KB to size s/g list (authored by gallatin).
isci: use maxphys rather than 128KB to size s/g list
Jan 7 2021, 5:48 PM
gallatin closed D28023: isci: use maxphys rather than 128KB to size s/g list.
Jan 7 2021, 5:48 PM
gallatin accepted D27683: iflib: ensure that tx interrupts are enabled and simplify txq_drain.
Jan 7 2021, 4:44 PM
gallatin accepted D27839: Add Chacha20-Poly1305 as a KTLS cipher suite..
Jan 7 2021, 4:34 PM
gallatin requested review of D28023: isci: use maxphys rather than 128KB to size s/g list.
Jan 7 2021, 4:26 PM

Jan 6 2021

gallatin accepted D27986: x86 tsc: use RDTSCP in preference of fence + RDTSC.

I tested the kernel part of this on a Netflix server with an EPYC 7502P using TSC-low. It saves roughly 2% CPU and tsc disappears entirely from profiling output, making profiling output match that from our Xeons much more closely.

Jan 6 2021, 3:18 PM

Dec 19 2020

gallatin closed D21636: Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain.
Dec 19 2020, 10:05 PM
gallatin committed rS368819: Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain.
Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain
Dec 19 2020, 10:05 PM
gallatin closed D21648: ktls: NUMA awareness.
Dec 19 2020, 9:46 PM
gallatin committed rS368818: Optionally bind ktls threads to NUMA domains.
Optionally bind ktls threads to NUMA domains
Dec 19 2020, 9:46 PM
gallatin accepted D21648: ktls: NUMA awareness.

Thanks so much for reviewing. I've made the changes you requested in the patch I'm about to commit.

Dec 19 2020, 9:38 PM

Dec 18 2020

gallatin updated the diff for D21648: ktls: NUMA awareness.

Rebase patch to r368767

Dec 18 2020, 9:03 PM
gallatin updated the diff for D21636: Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain.

Rebase patch as of r368767

Dec 18 2020, 8:57 PM

Dec 14 2020

gallatin accepted D27580: Use uintptr_t instead of unsigned long for integers holding pointers..
Dec 14 2020, 1:26 AM

Dec 13 2020

gallatin committed R9:8d2e6234bbf0: I've finally taken the plunge and become a committer. (authored by gallatin).
I've finally taken the plunge and become a committer.
Dec 13 2020, 7:59 PM

Dec 6 2020

mm awarded D21648: ktls: NUMA awareness a Like token.
Dec 6 2020, 9:39 PM
mm awarded D21636: Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain a Like token.
Dec 6 2020, 9:39 PM

Dec 4 2020

gallatin accepted D27448: Two commits to cleanup needless wmb() in nvme driver.

This looks correct to me.

Dec 4 2020, 6:35 PM
gallatin added a reviewer for D27448: Two commits to cleanup needless wmb() in nvme driver: gallatin.
Dec 4 2020, 6:35 PM

Nov 25 2020

gallatin added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

@gallatin I don't have the resources to fix it, so I'll just say of the things we were trying to do is get away from expert knob turning for common use cases, which I would lump a netapp filer and a CDN into for FreeBSD. iflib is supposed to not needlessly rearm interrupts while it is processing and there is no more work, so we should have some moderation of interrupts already or we need to add a callback into the framework so hardware like this can further modulate if necessary. I don't like bringing back a driver-only hack instead of holistically solving this. Nonetheless, if I am the only person that cares about that kind of thing I think my assumptions need to be retuned and I'll go away.

Nov 25 2020, 3:08 PM

Nov 24 2020

gallatin added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

A few more thoughts based on a conversation I had with somebody: In order for mbuf sorting to be effective, you need to have a large number of packets received per irq. Sorting a dozen mbufs is not useful when you're already aggregating 64 different connections, for example. So AIM (or very large irq colaescing timer/packet settings) are required to get enough packets per irq to make mbuf sorting worthwhile.

Nov 24 2020, 6:00 PM
gallatin accepted D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

Approving for just myself for now until the discussion regarding AIM / LRO is wrapped up.

Nov 24 2020, 3:25 PM
gallatin added a comment to D27344: Bring back AIM (Adaptive Interrupt Moderation) that was lost in IFLIB migration..

I would rather see this fixed by implementing LRO mbuf sorting in iflib, a la cxgbe and mlx5en. It's a small code change like https://freshbsd.org/commit/freebsd/src/317041. Would you be willing to look at this or discuss with intel to do it on your behalf?

Nov 24 2020, 3:23 PM

Nov 20 2020

gallatin added a comment to D26973: Switch C.UTF-8 as the default locales.

After this change, perl freaks out like this:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LC_ALL = (unset),
LANG = "C.UTF-8"

are supported and installed on your system.

perl: warning: Falling back to the standard locale ("C").
panic: locale.c: 4560: Could not change LC_CTYPE locale to C.UTF-8, errno=22

Is this expected? This is perl5-5.32.0

Thanks

Nov 20 2020, 3:55 PM

Nov 19 2020

gallatin added a comment to D26973: Switch C.UTF-8 as the default locales.

After this change, perl freaks out like this:

Nov 19 2020, 7:35 PM
gallatin accepted D27272: Add a manpage for kernel TLS..
Nov 19 2020, 2:14 PM

Nov 18 2020

gallatin closed D27188: LACP: When suppressing distributing, return ENOBUFS rather than ENETDOWN to preserve TCP conns.
Nov 18 2020, 2:56 PM
gallatin committed rS367797: LACP: When suppressing distributing, return ENOBUFS.
LACP: When suppressing distributing, return ENOBUFS
Nov 18 2020, 2:56 PM

Nov 16 2020

gallatin accepted D27245: savecore: uncompress cores.
Nov 16 2020, 10:53 PM

Nov 12 2020

gallatin added a reviewer for D27188: LACP: When suppressing distributing, return ENOBUFS rather than ENETDOWN to preserve TCP conns: rrs.
Nov 12 2020, 4:11 AM
gallatin requested review of D27188: LACP: When suppressing distributing, return ENOBUFS rather than ENETDOWN to preserve TCP conns.
Nov 12 2020, 4:11 AM

Nov 11 2020

gallatin accepted D27179: Free full mbuf chain when draining Tx.
Nov 11 2020, 4:21 PM

Oct 30 2020

gallatin added a comment to D26972: Make linux_errtbl[] more readable.

Isn't errno different in linux on different architectures? I know that at least alpha had a very different errno than x86..

Oct 30 2020, 3:18 AM

Oct 29 2020

gallatin accepted D27000: Add m_snd_tag_alloc() as a wrapper around if_snd_tag_alloc()..
Oct 29 2020, 12:19 AM
gallatin accepted D26999: Remove an extra if_ref()..
Oct 29 2020, 12:18 AM
gallatin accepted D26997: Store the new send tag in the right place..

Its in my branch, but not our main branch.

Oct 29 2020, 12:15 AM
gallatin accepted D26996: Use public interfaces to manage the nested rate limit send tag..
Oct 29 2020, 12:12 AM
gallatin accepted D26995: Call m_snd_tag_rele() to free send tags..
Oct 29 2020, 12:10 AM

Oct 22 2020

gallatin accepted D26691: Support hardware rate limiting (pacing) with TLS offload..
Oct 22 2020, 2:39 PM
gallatin accepted D26875: Save the current TCP pacing rate in t_pacing_rate..
Oct 22 2020, 2:38 PM

Oct 19 2020

gallatin accepted D26821: Add a kernel crypto driver using assembly routines from OpenSSL..
Oct 19 2020, 1:25 PM

Oct 12 2020

gallatin accepted D26729: Permit sending empty fragments for TLS 1.0..
Oct 12 2020, 8:02 PM

Oct 10 2020

gallatin accepted D26730: Add support to the KTLS OCF module for AES-CBC MTE ciphersuites..

Awesome. Thank you!

Oct 10 2020, 12:53 AM

Oct 9 2020

gallatin added a comment to D26729: Permit sending empty fragments for TLS 1.0..

Can we add some __predict_false() to aid the compiler (and the reader) to know these are rare?

Oct 9 2020, 11:59 PM
gallatin added a comment to D26691: Support hardware rate limiting (pacing) with TLS offload..

Looks good from mlx5_en_main.c's point of view.
@gallatin: Did you test this with mlx5en?

Oct 9 2020, 1:26 PM

Oct 8 2020

gallatin accepted D26691: Support hardware rate limiting (pacing) with TLS offload..
Oct 8 2020, 3:36 PM

Oct 6 2020

gallatin accepted D26690: Check if_capenable, not if_capabilities..
Oct 6 2020, 1:51 PM

Oct 5 2020

gallatin accepted D26689: Store the send tag type in the common send tag header..
Oct 5 2020, 9:19 PM

Oct 1 2020

gallatin accepted D26409: Add IP(V6)_VLAN_PCP to adjust 802.1 priority per-flow..

Thank you for making these changes.

Oct 1 2020, 4:10 PM

Sep 25 2020

gallatin added a comment to D26409: Add IP(V6)_VLAN_PCP to adjust 802.1 priority per-flow..

So, how about moving the PCP to become part of the inpcb, and use IPPROTO sockopts instead. Sounds like that may be the better approach; doing this in the socket struct was "just" the easiest way to get a proof-of-concept running for various reasons... While the PCP is part of the (extended) Ethernet Header, there are no setsockopt Protos that deal with data on that level...

Sep 25 2020, 9:00 PM
gallatin added inline comments to D26409: Add IP(V6)_VLAN_PCP to adjust 802.1 priority per-flow..
Sep 25 2020, 6:48 PM
gallatin added a reviewer for D26409: Add IP(V6)_VLAN_PCP to adjust 802.1 priority per-flow.: rrs.
Sep 25 2020, 6:42 PM
gallatin added inline comments to D26409: Add IP(V6)_VLAN_PCP to adjust 802.1 priority per-flow..
Sep 25 2020, 4:30 PM

Sep 21 2020

gallatin closed D26103: TCP: remove special treatment for hardware (ifnet) TLS.
Sep 21 2020, 1:05 AM

Sep 11 2020

gallatin added a comment to D26340: run clang-format over vtfontcvt.c.

Is there a file which is already perfectly style(9) compliant? Is it this file?

Sep 11 2020, 5:40 PM

Sep 7 2020

gallatin added a comment to D26340: run clang-format over vtfontcvt.c.

What command line arguments, or other clang format configuration was used here?

Sep 7 2020, 11:16 PM