Page MenuHomeFreeBSD

Add direct domain allocation functions and use them in busdma
ClosedPublic

Authored by jeff on Dec 20 2017, 12:57 AM.

Details

Summary

This is 100% plumbing.

It does three things:

  1. Add a malloc_domain(), uma_zalloc_domain(), and contigmalloc_domain() that allocate from the precise domain specified. This depends on earlier UMA changes in my branch.
  2. Creates a tag in every acpi pci host bridge that describes what domain the device lives in.
  3. Makes use of the domain for all busdma allocations on x86.

I need to add empty functions for non-x86 busdma implementations.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

jeff created this revision.Dec 20 2017, 12:57 AM
kib added inline comments.Dec 21 2017, 3:04 PM
x86/iommu/busdma_dmar.c
383 ↗(On Diff #36783)

Why internal dma implementation data needs to be domain-specific ?

In fact, dmar would benefit from allocating the IOMMU page table pages from the domain the DMAR belongs to, but I can handle this after the main NUMA pieces are finalized.

jeff added inline comments.Dec 21 2017, 7:43 PM
x86/iommu/busdma_dmar.c
383 ↗(On Diff #36783)

My assumption is the threads processing these lists are more likely to be on the local domain once we start binding driver threads. So it might help and it likely doesn't hurt.

My hope is that after we get this chunk of work in we can make small patches to improve locality of various things. per-cpu variables, for example. Thread stacks for bound threads possibly.

jhb accepted this revision.Jan 9 2018, 8:01 PM
jhb added inline comments.
dev/acpica/acpi_pcib_acpi.c
545 ↗(On Diff #36783)

You probably don't need the explicit PCI_DMA_BOUNDARY as the tag in the PCI bus layer will enforce that and deferring that to there means there's only one place that has to worry about that.

550 ↗(On Diff #36783)

This is probably fine (and the overall design is fine). I would be tempted to say that we should only provide a tag with a valid domain if _PXM actually exists (if acpi_parse_pxm(dev) returned a value >= 0 which would require making that function non-static), but that is a minor detail.

559 ↗(On Diff #36783)

We need to destroy the tag in this case.

This revision is now accepted and ready to land.Jan 9 2018, 8:01 PM
jeff marked an inline comment as done.Jan 9 2018, 9:51 PM
This revision was automatically updated to reflect the committed changes.