Add direct domain allocation functions and use them in busdma

[jeff]

Summary

This is 100% plumbing.

It does three things:

1. Add a malloc_domain(), uma_zalloc_domain(), and contigmalloc_domain() that allocate from the precise domain specified. This depends on earlier UMA changes in my branch.
2. Creates a tag in every acpi pci host bridge that describes what domain the device lives in.
3. Makes use of the domain for all busdma allocations on x86.

I need to add empty functions for non-x86 busdma implementations.

[kib]

x86/iommu/busdma_dmar.c

383 ↗

(On Diff #36783)

Why internal dma implementation data needs to be domain-specific ?

In fact, dmar would benefit from allocating the IOMMU page table pages from the domain the DMAR belongs to, but I can handle this after the main NUMA pieces are finalized.

[jeff]

x86/iommu/busdma_dmar.c

383 ↗

(On Diff #36783)

My assumption is the threads processing these lists are more likely to be on the local domain once we start binding driver threads. So it might help and it likely doesn't hurt.

My hope is that after we get this chunk of work in we can make small patches to improve locality of various things. per-cpu variables, for example. Thread stacks for bound threads possibly.

[jhb]

dev/acpica/acpi_pcib_acpi.c

545 ↗

(On Diff #36783)

You probably don't need the explicit PCI_DMA_BOUNDARY as the tag in the PCI bus layer will enforce that and deferring that to there means there's only one place that has to worry about that.

550 ↗

(On Diff #36783)

This is probably fine (and the overall design is fine). I would be tempted to say that we should only provide a tag with a valid domain if _PXM actually exists (if acpi_parse_pxm(dev) returned a value >= 0 which would require making that function non-static), but that is a minor detail.

559 ↗

(On Diff #36783)

We need to destroy the tag in this case.