Add netmap support to bhyve

[gnn]

Summary

This patch originally submitted by btw@. Updated to work on HEAD. Provides basic netmap support in the guest.

[btw]

I tried to test this patch on -HEAD, but it caused kernel panic. It seems that there is a bug in the kqueue support code of netmap. And I can reproduce the kernel panic with this simple program on -HEAD:

#include <stdio.h>
#include <assert.h>

#include <sys/types.h>
#include <sys/event.h>
#include <sys/time.h>

#ifndef NETMAP_WITH_LIBS
#define NETMAP_WITH_LIBS
#endif
#include <net/netmap_user.h>

static int
kq_add(int kq, int fd)
{
	struct kevent changes[1];
	int ret;

	EV_SET(&changes[0], fd, EVFILT_READ, EV_ADD, 0, 0, NULL);
	ret = kevent(kq, changes, 1, NULL, 0, NULL);
	assert(ret != -1);

	printf("[%s] success\n", __func__);

	return (ret);
}

static void
kq_wait(int kq)
{
	struct kevent events[1];
	int ret;

	ret = kevent(kq, NULL, 0, events, 1, NULL);
	assert(ret != -1);

	printf("[%s] success\n", __func__);
}

int main(void)
{
	const char *ifname = "vale0:vm1";
	struct nm_desc *d;
	int kq;

	d = nm_open(ifname, NULL, 0, 0);
	assert(d != NULL);

	kq = kqueue();
	assert(kq != -1);

	kq_add(kq, d->fd);
	kq_wait(kq);

	return (0);
}

And below is part of the crash summary:

freebsd dumped core - see /var/crash/vmcore.0

Fri Jan  8 22:19:39 CST 2016

FreeBSD freebsd 11.0-CURRENT FreeBSD 11.0-CURRENT #2 d1f6105(master)-dirty: Fri Jan  8 21:55:33 CST 2016     btw@freebsd:/usr/obj/root/freebsd/sys/GENERIC  amd64

......

panic: mutex nm_kn_lock not owned at /root/freebsd/sys/kern/kern_event.c:2073
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe011744c420
vpanic() at vpanic+0x182/frame 0xfffffe011744c4a0
panic() at panic+0x43/frame 0xfffffe011744c500
__mtx_assert() at __mtx_assert+0xbf/frame 0xfffffe011744c510
knlist_add() at knlist_add+0x20/frame 0xfffffe011744c540
netmap_kqfilter() at netmap_kqfilter+0x101/frame 0xfffffe011744c580
devfs_kqfilter_f() at devfs_kqfilter_f+0x81/frame 0xfffffe011744c5d0
kqueue_register() at kqueue_register+0x5bf/frame 0xfffffe011744c670
kqueue_kevent() at kqueue_kevent+0xc8/frame 0xfffffe011744c840
kern_kevent_fp() at kern_kevent_fp+0x99/frame 0xfffffe011744c890
kern_kevent() at kern_kevent+0x9f/frame 0xfffffe011744c8f0
sys_kevent() at sys_kevent+0x11c/frame 0xfffffe011744c9a0
amd64_syscall() at amd64_syscall+0x2db/frame 0xfffffe011744cab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe011744cab0
--- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80095bc7a, rsp = 0x7fffffffe988, rbp = 0x7fffffffe9e0 ---
KDB: enter: panic

......

[gnn]

I don't see that here either in the VM or outside of one. Here is the uname of my VM:

FreeBSD odevbox 11.0-CURRENT FreeBSD 11.0-CURRENT #0 2c6aacd(dtrace-gnn)-dirty: Wed Dec 16 04:55:40 EST 2015 root@odevbox:/usr/obj/mnt/minion/exported/github/CADETS/freebsd/sys/GENERIC-NODEBUG amd64

Are you seeing this just on your VM or on generic netmap on HEAD?

[btw]

I'm testing on the master branch of the repo cloned from https://github.com/freebsd/freebsd.git with GENERIC conf on a desktop computer. And I can also reproduce this panic in VirtualBox with the image downloaded from http://mirrors.aliyun.com/freebsd/snapshots/VM-IMAGES/11.0-CURRENT/amd64/20151217/FreeBSD-11.0-CURRENT-amd64-20151217-r292413.vhd.xz

[gnn]

If you're seeing this on an image without the patch then it's not the patch that's the problem. If that's the case then please open a PR against netmap and, if you like assign it to me but it ought to not block committing this work.

[gnn]

I have verified the bug you mention is unrelated to this patch. It happens on vanilla HEAD. I'd like to commit this patch and then look at the bug you mention.

[btw]

Yeah, the bug is unrelated to this patch. I'll open a PR against netmap and won't let it block committing work! And thank you very much for being willing to commit this patch! ^_^

I will try to revert netmap and test this patch!