Sadly but this failed testing even if I put DELAY(100).
I'll need to find out which request failed exactly.

Booted fine. All my PCI cards work fine.

o Place IOMMU to exact place where ACPI_DMAR was
o Remove IOMMU from i386/MINIMAL as we don't have it in i386/GENERIC

Remove unused headers

More style fixes:
o Rename header protection variables
o Make variables consistent:

struct smmu_unit is unit
struct smmu_domain is domain
struct smmu_ctx is ctx
struct iommu_unit is iommu
struct iommu_domain is iodom
struct iommu_ctx is ioctx

o Add locking around smmu_ctx_lookup() method
o ctx_free and domain_free methods return void
o Call to iommu_gas_fini_domain() and iommu_domain_fini() on domain deinitialization (iommu_domain_free())

Fix indentation

Address mmel's comments.

Rework the arm64 iommu interface so it could be used on other IOMMU engines in arm64 world, not only SMMU.
(Requested by mmel)

update copyright for smmu_quircks.c

Manage resident count for L3 pages of SMMU pmap

Set domain.end address (max guest addr + 1) to the VM_MAXUSER_ADDR

Fix comment

Rename iommu_smmu.c to smmu_iommu.c

The final patch to review

Address markj@ queries

Add default implementations of iommu_init/deinit

gicv3_iommu_deinit() added

o Check if entry is not NULL, otherwise return early
o Remove gicv3_its.c changes from this diff. They are moved to https://reviews.freebsd.org/D26878

Move iommu stubs to subr_bus_dma.c

In D26887#599735, @andrew wrote:

Would it make sense to move these to subr_bus_dma.c in the non-IOMMU case?

o Rename pmap_sremove_all() to pmap_sremove_pages()
o Remove unneeded check for L2_BLOCK in pmap_sremove_pages()

Add IOMMU quirks for all the devices listed in linux

Update AHCI quirk bitstring

Move the quirk to ahci driver

o Add a KASSERT
o Mark MSI fields in the iommu_domain as arch-specific

Regenerate

o Move iommu_get_ctx_domain(), iommu_get_dev_ctx() to iommu.h

In D26705#598296, @kib wrote:

I am curious. There is no arch restriction on the value of MSI base address ? System can map MSI page anywhere, and just programming the correct address into MSI address register would make it work ?

This is very non-x86ish.

o Put Andrew's copyright
o Remove ifdef IOMMU

We could also note that the amd64's IOMMU subsystem was splitted-out from amd64 DMAR support and is now generic, i.e. could be used by all architectures

Regenerate

Address kibs's notes

Address kib's notes:

• add iommu_types.h
• add md macro IOMMU_UNLOAD_SLEEP

context added

In D26705#595414, @br wrote:

In D26705#595261, @kib wrote:

So would adding a header iommu_msi.h with just msi-related prototypes enough ? I suspect that x86/msi.c can also get rid of iommu.h then.

I will try and see if it works

Manage MSI iommu pages

In D26705#595261, @kib wrote:

So would adding a header iommu_msi.h with just msi-related prototypes enough ? I suspect that x86/msi.c can also get rid of iommu.h then.

In D26705#595130, @kib wrote:

Convention for var.h is not suitable for your case IMO. We have signal.h and signalvar.h, where signal.h is usable by userspace and signalvar.h is for kernel private stuff.

Can you provide some enumeration of stuff you want to keep in iommu.h, and an explanation of why ?