Tested with an amd64 VM live migration and still can see the ARP on resume.

Thanks, LGTM. Some months require the use of an apostrophe, so it needs to be factored into the months names.

Thanks for the review, will get the extra newlines removed and this committed.

Fix comments by markj.

Colin, could you please add Elliot Reviewed by tag when committing, if this turns out to fix the issues you saw on AWS? Thanks.

Thanks, will adjust tomorrow and commit.

Thanks for taking a look. I will push this together with the generic part.

Superseded by D46122 & D46123.

Some questions. I mostly miss where g_io_new_uz() and the new set of helpers are introduced, as I'm not finding them in main git branch yet?

Thanks, I've incorporated the feedback.

Incorporate feedback.

Move part of the logic to be common between all possible arches, just leaving the initialization of the resource manager arch-specific.

In D44262#1020013, @ehem_freebsd_m5p.com wrote:

This is acceptable to me, but I believe the desire is have an okay by @markj. I believe resigning should make this more visible to @markj, rather than accepting.

In D44262#1010118, @ehem_freebsd_m5p.com wrote:

Most of what is presently here is definitely wrong. xen_intr_handle_upcall() retrieving the trapframe from curthread->td_intr_frame is definitely right. Issue is xen_intr_handle_upcall() needs to behave as a proper driver_filter_t for the XenPCI support. ARM(64) and RISC-V also need this, likely PowerPC would need similar. I do see two ways forward though:

1. I mentioned this before handing the patch for af610cabf1f off that I was uncertain about the handling of curthread->td_intr_nesting_level. As already mentioned, perhaps xen_arch_intr_handle_upcall() should be leaving the value alone?
2. I'm wondering about the indirect call to xen_intr_handle_upcall() via the XenPCI implementation. During that call curthread->td_intr_nesting_level would start out as 1, but would then end up as 2 when xen_intr_handle_upcall() calls xen_arch_intr_execute_handlers() => intr_execute_handlers() => intr_event_execute_handlers(). Perhaps xen_intr_handle_upcall() should be decrementing curthread->td_intr_nesting_level before it calls handlers?

I've noticed the load average seeming a bit on the high side on my ARM64 test machine. I'm unsure whether that might be this bug versus simply being a debugging kernel. I can state I've tried the second and it appeared to work.

In D44451#1016401, @markj wrote:

The current code however cannot be committed, because Elftoolchain objcopy segfaults when dong the conversion from elf32-i386 -> elf64-x86-64.

Is anyone actively working on switching? Maybe the objcopy bug is easy to fix.

In D44251#1009698, @ehem_freebsd_m5p.com wrote:

This is oddly similar to what Julien Grall did in 2015. For aarch64 the suggested range for the shared information page is available to the platform device. Perhaps the architecture should pass the struct resource * to xenpv?

xen_intr_handle_upcall() only requires the trap_frame because intr_execute_handlers() requires it, if the plan is to remove it from intr_execute_handlers() let's remove it from xen_intr_handle_upcall() at the same time.