Page MenuHomeFreeBSD

arm64: Elide coherent busdma maps
ClosedPublic

Authored by gallatin on Mon, Sep 28, 5:44 PM.
Tags
None
Referenced Files
F174735995: D60098.diff
Mon, Oct 5, 2:55 PM
F174667605: D60098.id.diff
Mon, Oct 5, 1:56 AM
F174657777: D60098.id187929.diff
Mon, Oct 5, 12:31 AM
F174654603: D60098.id187906.diff
Sun, Oct 4, 11:58 PM
Unknown Object (File)
Sun, Oct 4, 7:05 PM
Unknown Object (File)
Sat, Oct 3, 11:57 PM
Unknown Object (File)
Sat, Oct 3, 6:29 AM
Unknown Object (File)
Sat, Oct 3, 6:14 AM
Subscribers

Details

Summary

Avoid allocating per-transfer maps for coherent tags that cannot
bounce. Retain maps for cache synchronization, CCA realms, and KMSAN.
These un-used maps carry with them memory and cache miss overheads.

This saves close to 1% CPU on my tiny N1 setup serving ~80Gb/s of Netflix
traffic.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

FreeBSD/sys/arm64/arm64/busdma_bounce.c
980

Can this be called with a NULL map? Hmm, I guess it can. FWIW, x86 handles this in the header file so the check gets inlined sooner in sys/kern/subr_busdma_bounce.c. That is, x86 checks it in sys/x86/include/bus_dma.h:_bus_dmamap_waitok(). Presumably could do the same here?

  • handled null maps for _bus_dmamap_waitok() and bus_dmamap_unload() in the linline wrappers as suggested by @jhb
gallatin added inline comments.
FreeBSD/sys/arm64/arm64/busdma_bounce.c
980

Great idea.. Just implemented it. Thank you!

FreeBSD/sys/arm64/arm64/busdma_bounce.c
438–439

Couldn't this be merged into the if above? In the else case we return an error if *mapp == NULL.

779

Could we get here with buf being misaligned for the dma tag? If so it looks like might_bounce could return true so we write to nobounce_dmamap leading to a race.

gallatin added inline comments.
FreeBSD/sys/arm64/arm64/busdma_bounce.c
779

Good catch. I think the fix is to restrict map elision to coherent tags that accept any alignment. Thank you!

gallatin added inline comments.
FreeBSD/sys/arm64/arm64/busdma_bounce.c
779

Actually, this is not possible. When eliding the map, we check dmat->bounce_flags & (BF_COULD_BOUNCE | BF_COHERENT | BF_FORCE_MAP)) == BF_COHERENT , and BF_COULD_BOUNCE is set in bounce_bus_dma_tag_create() on tags with alignment > 1 or that cannot map all of memory.

simplified the map elision in bounce_bus_dmamap_create() by directly jumping to out when setting *mapp=NULL suggested by Andy

FreeBSD/sys/arm64/arm64/busdma_bounce.c
430

This else is unneeded. Everything to the out label are now part of the else

779

In that case would doing something like this here be useful? It lets us skip the might_bounce check & most of the while loop:

if (!_bus_dmamap_addsegs(dmat, map, curaddr, sgsize, segs, segp))
	bus_dmamap_unload(dmat, map);
	return (EFBIG); /* XXX better return value here? */
}
return (0);
870

We could also insert the call to _bus_dmamap_addsegs here & return.

Updated patch to implement 2 of Andy's suggestions:

  • remove the else bounce_bus_dmamap_create()
  • just call _bus_dmamap_addsegs in bounce_bus_dmamap_load_phys
gallatin added inline comments.
FreeBSD/sys/arm64/arm64/busdma_bounce.c
870

I don't think that works here. It works in load_phys because the addr is phys contig. Here we have a virtual addr which might not be phys contig, and we need the pmap_extract() below to do virt->phys.

This revision is now accepted and ready to land.Fri, Oct 2, 3:10 PM
This revision was automatically updated to reflect the committed changes.