Thanks for splitting your initial commit! Note that this can also be split into smaller commits to make it easier to read: add check_iov_len, update buf_to_iov and replace seek + truncate by split. While I'm personally preferring splitting the commits, I'd be fine keeping it as is too.

In D53220#1216044, @rosenfeld_grumpf.hope-2000.org wrote:

In D53220#1215995, @corvink wrote:

It would make it easier to review and merge when each point gets it own commit.

Actually, all of this was done together with D53223, D53222, and D53221. I've moved this part out and ahead of the other changes as I figured they'd needlessly complicate the review of the larger functional changes.

It may seem as a bunch of individual changes lumped together, but I've really just listed what this change does. Except perhaps for the style cleanups (which I'm not sure are well received in FreeBSD anyway) and the 1-line change to enable indirect descriptors, I'm not sure much is gained if I try to split it up further as all of this was really developed and tested together.

Anyway, if you insist that I split it up, I will of course try to do so.

@kib This can be closed. The issues was fixed by D52968.

• indicate support for indirect descriptors
• simplify iov functions and let them work in-place, preventing unnecessary allocations and memcpy() calls in hot code paths
• preallocate all I/O requests on all queues, taking most allocations out of hot code paths
• check for I/O request validity as early as possible, and return illegal requests immediately
• check lun format
• check all allocations for success and handle failures gracefully
• add a few more DPRINTFs
• style cleanups

Please descripe the bug and expected behavior in more detail in the commit message.

Fixed by D52968

In D52968#1214028, @markj wrote:

In D52968#1213618, @corvink wrote:

In D52968#1213405, @markj wrote:

In D52968#1212719, @corvink wrote:

It doesn't look like it fully freezes. However, it gets extremely slow. The circle of the Windows boot loader spins very slowly. Don't know if it will ever reach the desktop.

What output do you see if you run procstat -kk <bhyve PID> while this is happening?

Almost always one vCPU hangs in vm_handle_rendezvous. The output is from a single VM run. I've called procstat a few times within a few seconds:

Could you please try the updated patch?

In D52968#1213405, @markj wrote:

In D52968#1212719, @corvink wrote:

It doesn't look like it fully freezes. However, it gets extremely slow. The circle of the Windows boot loader spins very slowly. Don't know if it will ever reach the desktop.

What output do you see if you run procstat -kk <bhyve PID> while this is happening?

It doesn't look like it fully freezes. However, it gets extremely slow. The circle of the Windows boot loader spins very slowly. Don't know if it will ever reach the desktop.

In D52781#1208670, @jon_xyinn.org wrote:

I think there may still be a relationship, jbo, since I've been having a different experience than you. However for now I'll try out some other bisecting before coming back to this. I'm going to test out switching out the "nvme" emulation for my drive to "ahci-hd" and see if that has any stability effect. I know people have also experienced issues with the nvme emulation before.

• only unlock vcpu when necessary
• avoid uneccessary sleeps
• fix some style issues

Hi,
I'm able to reproduce the issue and was able to fix it: https://reviews.freebsd.org/D52781

I'm not sure but it might be possible to use an RB tree [1] to simplify creating a list, sorting it and searching for elements (RB_NFIND).

In D51892#1186266, @jhb wrote:

Hmmm, do we not support INTx interrupts for passthrough in general? I guess we don't. There's a chance some guest OS might try to use INTx instead of MSI which won't work. Do you have more context on what is requiring this? Is it a driver in a Windows guest, or some other OS?

Setting every single cpuid value seems to be a bunch of work for user. Especially, as user have to set every single bit correctly. What's the way forward for this? How will bhyve receive some default cpuid values?

This should be merged into the previous commit D51552 shouldn't it?

Could you please describe a use case in the commit message?

• rename len to size to match common naming in pci_passthru.c

• use uint64_t instead of vm_* types
• account for new handler interface (including baridx)

• use uint64_t instead of vm_* types

• use uint64_t types instead of vm_* types
• add baridx parameter to read/write handler
  • this parameter can be used in the future to e.g. add a generic PCI config mirror in BAR space which is quite common according to QEMU [1]

@markj @jhb Any additional comments?

• fix build

• fix style issue
• use correct offset for 64 bit BDSM read/writes

• fix style issues
• make use of trunc_page and round_page

• fix style issues
• make global variables const

Becomes obsolete by D50806 and D50807.

• rebase onto main

• rebase onto main

• rebase onto main

• rebase onto main

• make use of igd_ops

• rebase onto main