Fix numerous refcount bugs in multicast leading to use after free of ifnet, if_multiaddr, and in_mcast, as well as null derefs
There are 4 different things that can be freed independently of each other
ifnet (interface structure), if_multiaddr (interface multicast address), in_mcast (multicast address), pcb (protocol control block for connections). If one goes away we have to make sure to not reference it from anywhere else. Conversely anything that points to one of these has to hold a reference. Multicast isn't really well thought out for multithreading.
a) in_getmulti implicitly relies on the caller to add a reference when a new multicast address is created and *not* do so when it's an existing one. Breaking this contract has led to pf and carp causing use after free and could be causing in_multi leaks.
b) If a link layer address is freed before the network layer address, there can be a null deref. Clear the pointer in the network address.
c) the if_multiaddr holds a reference to an ifnet, to insure the ifnet doesn't go away before the multiaddr does, acquire a reference on create and release the reference on free and then clear the pointer to the ifnet
d) if the ifnet has gone away, the ll_ifma pointer is no longer valid
e) release the ifma reference to the ifnet on first delete (possibly incorrect)
f) clear the pointer to the in_mcast in the if_multiaddr when we release the reference in igmp_ifdetach to avoid stale references
g) similarly clear the in_mcast pointer in the ifma in purgemaddrs
h) in inp_join_group don't bump imo_num_memberships until we have actually assigned the mcast address to the array entry, thus avoid a null pointer deref with an invalid value
i) simplify freeing in the error case in carp_multicast_setup
j) ip_moptions handling isn't so performance critical that we have to use an unsigned short for num memberships, thus leading silently to underflow
k) in6_getmulti has to bump the ifma reference once inm points to it
l) removing members in v6 needs to be separate from disconnecting the in6_mcast
m) in6p_join_group would try to unlock the pcb without it held in the error path
n) any foreach loop that can lead to an entry being deleted from the list its iterating over needs to use the _SAFE macro (mld_ifdetach)
pkg install mdnsresponder sysrc mdnsd_enable=YES service mdnsd start ifconfig vtnet0 delete # or whatever iface egress/dhclient is running on ifconfig epair create ifconfig epair0a 0/24 up ifconfig epair0a destroy # panic
inm: 0xfffff8005a31d700 refcount: 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0091f3ee50
in_purgemaddrs() at in_purgemaddrs+0xce/frame 0xfffffe0091f3ee90
in_ifdetach() at in_ifdetach+0x83/frame 0xfffffe0091f3eeb0
if_detach_internal() at if_detach_internal+0x7ee/frame 0xfffffe0091f3ef30
if_detach() at if_detach+0x3d/frame 0xfffffe0091f3ef50
epair_clone_destroy() at epair_clone_destroy+0x117/frame 0xfffffe0091f3efa0
if_clone_destroyif() at if_clone_destroyif+0x175/frame 0xfffffe0091f3eff0
if_clone_destroy() at if_clone_destroy+0x205/frame 0xfffffe0091f3f040
ifioctl() at ifioctl+0x4ee/frame 0xfffffe0091f3f110
kern_ioctl() at kern_ioctl+0x28a/frame 0xfffffe0091f3f180
sys_ioctl() at sys_ioctl+0x15d/frame 0xfffffe0091f3f250
amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0091f3f370
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0091f3f370
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x80047008a, rsp = 0x7fffffffe2f8, rbp = 0x7fffffffe310 ---
freeing 0xfffff8005a31d700
root@beastie3:~ #
Fatal trap 9: general protection fault while in kernel mode
cpuid = 13; apic id = 0d
instruction pointer = 0x20:0xffffffff80d648e7
stack pointer = 0x28:0xfffffe0000538170
frame pointer = 0x28:0xfffffe00005381b0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 0 (softirq_13)
trap number = 9
panic: general protection fault
cpuid = 13
time = 1556067304
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000537e80
vpanic() at vpanic+0x19d/frame 0xfffffe0000537ed0
panic() at panic+0x43/frame 0xfffffe0000537f30
trap_fatal() at trap_fatal+0x394/frame 0xfffffe0000537f90
trap() at trap+0x6c/frame 0xfffffe00005380a0
calltrap() at calltrap+0x8/frame 0xfffffe00005380a0
--- trap 0x9, rip = 0xffffffff80d648e7, rsp = 0xfffffe0000538170, rbp = 0xfffffe00005381b0 ---
in_leavegroup_locked() at in_leavegroup_locked+0xa7/frame 0xfffffe00005381b0
inp_freemoptions() at inp_freemoptions+0x150/frame 0xfffffe0000538200
in_pcbfree_deferred() at in_pcbfree_deferred+0x16f/frame 0xfffffe0000538250
epoch_call_task() at epoch_call_task+0x1aa/frame 0xfffffe00005382b0
gtaskqueue_run_locked() at gtaskqueue_run_locked+0xf9/frame 0xfffffe0000538300
gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe0000538330
fork_exit() at fork_exit+0x84/frame 0xfffffe0000538370
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000538370
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
[ thread pid 0 tid 100039 ]
Stopped at kdb_enter+0x3b: movq $0,kdb_whyWill rebase to 20070 when it goes in.