CARP: proper loopback detection and few improvements
Needs ReviewPublic
Actions

Authored by rozhuk.im-gmail.com on Feb 3 2020, 2:57 AM.

Details

Reviewers

glebius
emaste
melifaro
mav

Group Reviewers

network

Summary

Loop back detection.

This PR reverts 310847 (first half).
For loop back detection now used counter field of CARP packet, that was in CARP to resist replay packets.
Check is simple now: carp_counter from packet compares with carp_counter last send packet.

Add counter for loop back detection.

CARP packets with VHID = 0 now dropped as "invalid VHID" before any other checks.

carp_version != CARP_VERSION || ch->carp_authlen != CARP_AUTHLEN - now is second check.

Add definition for CARP_AUTHLEN.

Removed sc_init_counter, that fix carp_counter - it always generate from random.

Add carps_badif and carps_dups to netstat.

Few micro changes:

sizeof(sc->sc_counter) -> (sizeof(uint32_t) * 2)
snprintf(subsys, IFNAMSIZ+5, -> snprintf(subsys, sizeof(subsys),

PS: also I see that:

	/* check if received on a valid carp interface */
	if (m->m_pkthdr.rcvif->if_carp == NULL) {
		CARPSTATS_INC(carps_badif);
		CARP_DEBUG("%s: packet received on non-carp interface: %s\n",
		    __func__, m->m_pkthdr.rcvif->if_xname);
		m_freem(m);
		return (IPPROTO_DONE);
	}

only in carp6_input(), probably carp_input() need this check too.

Test Plan

Create file mc_loop.sh and chmod +x it.

#!/bin/sh

# Copyright (c) 2020 Rozhuk Ivan <rozhuk.im@gmail.com>
# All rights reserved.
# 
# Subject to the following obligations and disclaimer of warranty, use and
# redistribution of this software, in source or object code forms, with or
# without modifications are expressly permitted by Whistle Communications;
# provided, however, that:
# 1. Any and all reproductions of the source or object code must include the
#    copyright notice above and the following disclaimer of warranties; and
# 2. No rights are granted, in any manner or form, to use Whistle
#    Communications, Inc. trademarks, including the mark "WHISTLE
#    COMMUNICATIONS" on advertising, endorsements, or otherwise except as
#    such appears in the above copyright notice or in the software.
# 
# THIS SOFTWARE IS BEING PROVIDED BY WHISTLE COMMUNICATIONS "AS IS", AND
# TO THE MAXIMUM EXTENT PERMITTED BY LAW, WHISTLE COMMUNICATIONS MAKES NO
# REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, REGARDING THIS SOFTWARE,
# INCLUDING WITHOUT LIMITATION, ANY AND ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.
# WHISTLE COMMUNICATIONS DOES NOT WARRANT, GUARANTEE, OR MAKE ANY
# REPRESENTATIONS REGARDING THE USE OF, OR THE RESULTS OF THE USE OF THIS
# SOFTWARE IN TERMS OF ITS CORRECTNESS, ACCURACY, RELIABILITY OR OTHERWISE.
# IN NO EVENT SHALL WHISTLE COMMUNICATIONS BE LIABLE FOR ANY DAMAGES
# RESULTING FROM OR ARISING OUT OF ANY USE OF THIS SOFTWARE, INCLUDING
# WITHOUT LIMITATION, ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
# PUNITIVE, OR CONSEQUENTIAL DAMAGES, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES, LOSS OF USE, DATA OR PROFITS, HOWEVER CAUSED AND UNDER ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF WHISTLE COMMUNICATIONS IS ADVISED OF THE POSSIBILITY
# OF SUCH DAMAGE.
#
# Author: Rozhuk Ivan <rozhuk.im@gmail.com>
#


# This script return multicast/broadcast packets back.
# IFNAME - network interfae name



IF_NAME="$2"
PATTERN_CARP_FLT="ether[0] & 1 = 1 and (ether[0:4] != 0xffffffff or ether[4:2] != 0xffff) and ip[9] = 112"
# allways NOT MATCH
BPFPROG_PASSTROUTH="bpf_prog_len=1 bpf_prog=[ { code=6 jt=0 jf=0 k=0 } ]"


usage_msg()
{
	echo "usage: start|stop IFNAME"
	echo "This will enable multicast/broadcast loopback on IF."
}


if [ -z "${IF_NAME}" ]; then
	usage_msg
	return 1
fi
if ! ifconfig "$IF_NAME" > /dev/null 2>&1 ; then
	usage_msg
	echo "Invalid upstream interface: ${IF_NAME}"
	return 1
fi


case "$1" in
start)
	echo "start bridging on ${IF_NAME}"

	# load modules
	kldload ng_ether > /dev/null 2>&1
	kldload ng_bpf > /dev/null 2>&1
	kldload ng_hub > /dev/null 2>&1

	BPFPROG_CARP_FLT=$( tcpdump -i ${IF_NAME} -s 65535 -ddd ${PATTERN_CARP_FLT} | \
		( read len ; \
		echo -n "bpf_prog_len=$len " ; \
		echo -n "bpf_prog=[" ; \
		while read code jt jf k ; do \
		  echo -n " { code=$code jt=$jt jf=$jf k=$k }" ; \
		done ; \
		echo " ]" ) )

	# create and connect nodes
	ngctl mkpeer ${IF_NAME}: bpf lower lower
	ngctl name ${IF_NAME}:lower ${IF_NAME}-bpf
	ngctl connect ${IF_NAME}: ${IF_NAME}-bpf: upper upper
	ngctl mkpeer ${IF_NAME}-bpf: hub h0 h0
	ngctl name ${IF_NAME}-bpf:h0 ${IF_NAME}-hub
	ngctl connect ${IF_NAME}-bpf: ${IF_NAME}-hub: h1 h1
	ngctl connect ${IF_NAME}-bpf: ${IF_NAME}-hub: h2 h2

	# configure BPF node
	ngctl msg ${IF_NAME}-bpf: setprogram { thisHook=\"upper\" ifMatch=\"h0\" ifNotMatch=\"lower\" ${BPFPROG_CARP_FLT} }
	ngctl msg ${IF_NAME}-bpf: setprogram { thisHook=\"lower\" ifMatch=\"\" ifNotMatch=\"upper\" ${BPFPROG_PASSTROUTH} }
	ngctl msg ${IF_NAME}-bpf: setprogram { thisHook=\"h1\" ifMatch=\"\" ifNotMatch=\"upper\" ${BPFPROG_PASSTROUTH} }
	ngctl msg ${IF_NAME}-bpf: setprogram { thisHook=\"h2\" ifMatch=\"\" ifNotMatch=\"lower\" ${BPFPROG_PASSTROUTH} }

	;;
stop)
	echo "stop multicast loopback on ${IF_NAME}"

	# remove hooks and nodes
	ngctl rmhook ${IF_NAME}: lower
	ngctl rmhook ${IF_NAME}: upper
	ngctl shutdown ${IF_NAME}-bpf:

	# unload modules
	#kldunload ng_bpf > /dev/null 2>&1
	#kldunload ng_ether > /dev/null 2>&1
	;;
*)
	usage_msg
esac


return 0

Configure CARP on some if:

ifconfig lan0 vhid 111 advskew 100 pass '123' alias 192.168.14.136/25

Ensure that it works

tcpdump -n -e -vvvvvvvvvv -T carp -i lan0 "ether[0] & 1 = 1 and (ether[0:4] != 0xffffffff or ether[4:2] != 0xffff) and ip[9] = 112"
04:42:38.944360 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=3380771757336744709
04:42:40.350677 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=3590934860648413426
04:42:41.812608 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=7614681408549585423
04:42:43.223786 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=3330805397649727691
04:42:44.634935 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=8117757783970461155
04:42:46.037239 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=8628093896405314600

(counter - is random)

ifconfig lan0 - MASTER.

Run (enable multicast loop back for CARP):

./mc_loop.sh start lan0

check dmesg log for:

carp: 111@lan0: MASTER -> BACKUP (more frequent advertisement received)
carp: 111@lan0: BACKUP -> MASTER (master timed out)
carp: 111@lan0: MASTER -> BACKUP (more frequent advertisement received)
carp: 111@lan0: BACKUP -> MASTER (master timed out)

and ifconfig lan0 - BACKUP.

netstat -sp carp
carp:
	97 packets received (IPv4)
	0 packets received (IPv6)
		0 packets discarded for wrong TTL
		0 packets shorter than header
		0 discarded for bad checksums
		0 discarded packets with a bad version
		0 discarded because packet too short
		0 discarded for bad authentication
		0 discarded for bad vhid
		0 discarded because of a bad address list
	2107 packets sent (IPv4)
	0 packets sent (IPv6)
		0 send failed due to mbuf memory error

After apply patch and rebuild kernel and netstat, tcpdump will show:

05:06:22.356929 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=6878480634282634828
05:06:23.763746 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=6878480634282634829
05:06:25.176950 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=6878480634282634830
05:06:26.592383 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=6878480634282634831
05:06:28.004406 00:00:5e:00:01:6f > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    172.16.0.126 > 224.0.0.18: carp 172.16.0.126 > 224.0.0.18: CARPv2-advertise 36: vhid=111 advbase=1 advskew=100 authlen=7 counter=6878480634282634832

(counter - not random any time)

ifconfig lan0 - now MASTER.

netstat -sp carp
carp:
	150 packets received (IPv4)
	0 packets received (IPv6)
		0 packets discarded for wrong interface
		0 packets discarded for wrong TTL
		0 packets shorter than header
		0 discarded for bad checksums
		0 discarded packets with a bad version
		0 discarded because packet too short
		0 discarded for bad authentication
		0 discarded for bad vhid
		0 discarded because of a bad address list
		150 discarded because packet is duplicate
	181 packets sent (IPv4)
	0 packets sent (IPv6)
		0 send failed due to mbuf memory error

(150 discarded because packet is duplicate!!!)