Thanks, I love the simplicity of this.

A few minor comments, I've looked mainly at hwpmc_cmn600.c so far.

Is there a review for a dmc620 driver still to come?

Seems good to me. One thing missing is loader_hints.disabled needs an entry in the config(5) man page.

This seems fine, these drivers use the xen_intr_ interfaces rather than anything contained in the header. Just one question about pcifront.c.

Log events as BOOTTRACE rather than RUNTTRACE. We will use the switch to multi-user mode as the mark between boot and run-time tracing.

Hi Colin, thanks for the comments and apologies that it has taken some time for me to update this patch.

Simplify the trace calls somewhat with the BOOTTRACE() macro. Now, printf-like arguments are supported.

Rebase and update after a few rounds of cleanups and testing.

In D30006#731742, @ehem_freebsd_m5p.com wrote:

@mhorne I think you got mixed up on the direction this is going.

Presently there are two paths which lead to xen_intr_handle_upcall():

1. apic_vector.S (x86 assembly/raw interrupt vectors) => xen_intr.c/xen_intr_handle_upcall()
2. xenpci.c/BUS_SETUP_INTR() => xenpci.c/xenpci_intr_filter() => xen_intr.c/xen_intr_handle_upcall()

I want to rearrange this sequence to add a third caller:

1. apic_vector.S (interrupt) => x86/xen_arch_intr.c/xen_arch_intr_handle_upcall() => xen_intr.c/xen_intr_handle_upcall()
2. xenpci.c/BUS_SETUP_INTR() => xen_intr.c/xen_intr_handle_upcall()
3. arm64/xen_arch_intr.c/bus_setup_intr() (D29875) => xen_intr.c/xen_intr_handle_upcall()

There are two reasons for changing like this. First, some bits of x86-specific code are still in xen_intr_handle_upcall(); notably the interrupt counter increment, critical_enter()/critical_exit() calls and lapic_eoi() call. Second, in theory a device_filter_t function has the capability to report stray interrupts, but any such reporting is lost with the present signature of xen_intr_handle_upcall().

In D30006#731339, @ehem_freebsd_m5p.com wrote:

Due to exploring elsewhere the issue of the struct trapframe has shown. I'm uncertain how to handle correctly. In its present form, this should behave identically to the previous version. Issue is there is a problem with the present form and I'm wondering whether the present form is correct.

The behavior of passing the trap frame as the argument is undocumented behavior for bus_setup_intr(). I've also encountered several mentions of peripheral interrupt controllers passing NULL trap frames.

Okay, let's take a step back here.

Add the hex prefix.

Handle !VIRT_IS_VALID for the usermode case.

In D31209#729707, @markj wrote:

Add a test case to kern_copyin.c.

Thanks.

It seems the hang cannot be triggered with a call to write(2), so use fcntl(2) as is done in the PR.

Any idea why?

Okay, I was able to boot this on a few devices for arm, arm64, and riscv.

Seems fine to me. Is this exhaustive or is it just the drivers you encountered?

Overall, I don't like the idea of making changes to working code for the benefit of a feature that is incomplete and has never been enabled. IMO any action on INTR_SOLO requires a more thorough set of changes + testing to make it production ready. It would probably be equally fine to remove it at this point.

Handle jrtc27's suggestion, add VIRT_IS_VALID() to vmparam.h.

Apply the change to routines in support.S as well.

Drop non-functional 'cycles' alias in favor of D32197.

@ehem_freebsd_m5p.com I committed the KASSERT alongside D30280, so this can be closed (but you will need to do it).

In D31494#720213, @domagoj.stolfa_gmail.com wrote:

Thanks for writing this up @mhorne, much appreciated! I've left a few comments from my experience of setting it up with QEMU + KVM today and the confusion I had while reading the existing documentation, as well as the confusion that I would probably have while reading this.

I wonder if it might make sense to explicitly state that the procedure for virtualization gdb stubs may be different depending on which hypervisor/emulator is being used, and then in follow-up commits (by you or anyone else really) document the procedure for each given hypervisor?

• Change the flow of the example so that it doesn't require DDB
• Mention that GDB is enabled in GENERIC on main
• Add a tip about local vars and optimization levels
• Other small wording/formatting improvements

Seems straightforward. Are there any manpages that need updating? Should FreeBSD sources still prefer sys/endian.h going forward, and is that mentioned anywhere?

Thank you for handling this. One small note which you can fix before commit. Don't forget to bump .Dd, of course :)

Why should the DMAP alias be writable? Its protections are supposed to be kept in sync with the corresponding KVA page. You add a comment to this effect in D32026, but I think pmap_change_props_locked() is still missing the code that actually updates the DMAP protections (at least I don't see it).

Fix the assertions, first cut at a more detailed block comment.

Add helpers to eliminate code churn outside of minidump_machdep.c.

Make the state pointer const.

Move function to vm_phys.c, rename it to vm_phys_is_dumpable().

Enabling this will limit the ability to set breakpoints or fbt probes on preloaded modules. On amd64 there is a global write-protect bit that is disabled before these operations, but on arm64 we will need something more fine-grained. Do you have plans to handle this?

Gentle ping.

Looks fine to me. I would suggest keeping the KASSERT though, it is free and makes the point of failure more obvious if this condition did occur.

This is mostly mechanical. state->dump_bitset should be used inside each minidump_machdep.c, and vm_page_dump everywhere else. The one exception to this is powerpc's dumpsys_scan_pmap() method, which is extended to also accept a bitset argument.