Differential D55492
virtio_net: Use bus_dma for rxq/txq buffers Authored by sarah.walker2_arm.com on Tue, Feb 24, 4:29 PM. Tags Referenced Files
Details While the majority of virtio platforms will be fully coherent, some may The virtio buffers are marked as coherent; this should ensure that sync Sponsored by: Arm Ltd
Diff Detail
Event TimelineComment Actions A quick test on RISC-V IOMMU gives me this issue: vtnet0: unable to set MAC address vtnet0: cannot disable promiscuous mode vtnet0: cannot disable all-multicast mode vtnet0: error setting host MAC filter table panic: alignment failed: ctx 0xffffffd0010d0e80 start 0x20c000 offset 82 align 0x4 Comment Actions vtnet0: unable to set MAC address vtnet0: cannot disable promiscuous mode vtnet0: cannot disable all-multicast mode vtnet0: error setting host MAC filter table panic: alignment failed: ctx 0xffffffd0010d0e80 start 0x20c000 offset 82 align 0x4 cpuid = 0 time = 1772019624 KDB: stack backtrace: db_trace_self() at db_trace_self db_fetch_ksymtab() at db_fetch_ksymtab+0x142 kdb_backtrace() at kdb_backtrace+0x2c vpanic() at vpanic+0x16e panic() at panic+0x26 iommu_domain_fini() at iommu_domain_fini+0x36a iommu_is_buswide_ctx() at iommu_is_buswide_ctx+0x806 bus_dmamap_load_mbuf_sg() at bus_dmamap_load_mbuf_sg+0x60 vtmmio_attach() at vtmmio_attach+0x488c vtmmio_attach() at vtmmio_attach+0x6656 vtmmio_attach() at vtmmio_attach+0x5652 ether_ioctl() at ether_ioctl+0x1a0 vtmmio_attach() at vtmmio_attach+0x5844 in_control_ioctl() at in_control_ioctl+0x9ac rtnl_ifaces_destroy() at rtnl_ifaces_destroy+0x1a1e rtnl_register_messages() at rtnl_register_messages+0x2a0 nl_send() at nl_send+0x3ca nl_taskqueue_handler() at nl_taskqueue_handler+0x34e taskqueue_run() at taskqueue_run+0x1f0 taskqueue_thread_loop() at taskqueue_thread_loop+0xd4 fork_exit() at fork_exit+0x68 fork_trampoline() at fork_trampoline+0xa KDB: enter: panic [ thread pid 0 tid 100062 ] Stopped at kdb_enter+0x3a: sd zero,560(s1) Comment Actions note function names in backtrace sometimes wrong, so it is more like bus_dmamap_load_mbuf_sg() -> iommu_bus_dmamap_load_buffer() -> iommu_bus_dmamap_load_something() -> iommu_bus_dmamap_load_something1() -> panic Comment Actions works with that. There are some warnings, but ping works vtnet0: unable to set MAC address vtnet0: cannot disable promiscuous mode vtnet0: cannot disable all-multicast mode vtnet0: error setting host MAC filter table vtnet0: link state changed to UP vtnet0: error setting host MAC filter table Starting Network: lo0 vtnet0. Comment Actions Apologies, not sure what I was thinking with the previous change. Can you try the current change please? Comment Actions It works but shows this warnings. Also vtnet does not seem to work in iommu bypass mode (hangs after "lo0: link state changed to UP"). lo0: link state changed to UP vtnet0: unable to set MAC address vtnet0: cannot disable promiscuous mode vtnet0: cannot disable all-multicast mode vtnet0: error setting host MAC filter table vtnet0: link state changed to UP vtnet0: error setting host MAC filter table Starting Network: lo0 vtnet0. Comment Actions looks like some part of vtnet still tries to pass physical addresses instead of bus dma addresses, so I get page faults riscv_fq_intr riscv_iommu0: Fault: event 0xd received: Read page fault riscv_iommu0: iotval 0x80e4940a iotval2: 0x0 riscv_iommu0: Fault: event 0xd received: Read page fault riscv_iommu0: iotval 0x80e4940a iotval2: 0x0 riscv_iommu0: Fault: event 0xd received: Read page fault riscv_iommu0: iotval 0x80e4940d iotval2: 0x0 riscv_iommu0: Fault: event 0xd received: Read page fault riscv_iommu0: iotval 0x80e4940d iotval2: 0x0 riscv_iommu0: Fault: event 0xf received: Write/AMO page fault riscv_iommu0: iotval 0x80e4940f iotval2: 0x0 riscv_iommu0: Fault: event 0xf received: Write/AMO page fault riscv_iommu0: iotval 0x80e4940f iotval2: 0x0 vtnet0: cannot disable all-multicast mode Comment Actions I'm getting a little confused now. Does vtnet work on RISC-V IOMMU without my changes? Comment Actions
Yes, it did work in Bypass mode, and works with this patch as well in bypass. also please keep lines 80 characters long per style(9) as some don't fit my terminal :) Comment Actions Could you try https://reviews.freebsd.org/D55564 (in addition to this change set) please? Comment Actions
thanks, it works! Only small issue with large tag alignment Comment Actions Hi, First up - the alignment should be what the hardware supports, not what the ethernet type supports. Eg, if the virtio API supports 1 byte alignment and it's not hugely inefficient to do so then that's what you should create the DMA tag with. On transmit - typically network drivers will first try the mbuf load, and if the load fails due to alignment / segment counts then it'll attempt to re-linearise the mbuf and load THAT, and if THAT fails it'll fail to transmit. The current code here is still doing m_defrag() before doing the load_phys call, which will linearise it and likely eliminate any benefit of using scatter/gather DMA support. Also, I've published some QEMU recipes for ppc64 pseries which use virtio for net/disk - https://wiki.freebsd.org/powerpc/QEMU/Pseries9 and https://wiki.freebsd.org/powerpc/QEMU/Pseries8 . I don't (yet) have pseries hardware VMs at home so I can't test it myself on real hardware, but it should be pretty easily testable in emulation. |