fix global demotion counter to VRRP advertisements

[olivier]

Summary

When net.inet.carp.preempt=1 and a physical interface goes down, the global V_carp_demotion counter is incremented. For CARP this was already reflected in outgoing advertisements via DEMOTE_ADVSKEW(), but VRRP sent the raw sc_vrrp_prio unchanged, so demotion had no effect.

Add DEMOTE_VRRP_PRIO(), a macro analogous to DEMOTE_ADVSKEW(): it subtracts V_carp_demotion from the configured priority and clamps the result to [0, 254]. Priority 0 is VRRPv3's "resign" signal and causes backups to preempt immediately. Priority 255 (IP address owner) is never demoted.

Apply the macro in three places in vrrp_input_c():

• when building outgoing advertisements, so a backup can preempt;
• in the MASTER receive path, so a demoted master yields to a backup whose priority exceeds the demoted value;
• in the BACKUP preempt check, so a demoted node does not immediately preempt back.

Addied a test that with net.inet.carp.preempt=1, bringing down one physical interface demotes all VRRPv3 instances on the host, causing a backup to take over on the remaining interfaces. Also verify that the higher-priority router reclaims MASTER once the interface is restored.

Test Plan

cd /usr/tests
sudo kyua test sys/netinet/carp:vrrp_preempt

[bms]

Just my 2c: I think we should be discouraging users from implementing CARP at all. The political history of CARP is contentious because of the patent rights Cisco had around VRRP which caused the OpenBSD developers to implement CARP in the first place. Tony Li was willing to work from within Cisco to ensure the patent was freely licensed to the OpenBSD guys, but they were not willing to work with him, so we got two protocols which collide when used on the same LAN segment. These days the patents surrounding VRRP have either expired or been once again freely granted, to the best of my knowledge, and as @olivier is probably well aware. There are also technical reasons for choosing one over the other (mostly to do with broadcast vs multicast), and CARP wound up being a garden rake on the floor when I was merging IGMPv3 and MLDv2 back in the late 00s. So it is no wonder projects like HAProxy have chosen to go the VRRP route.

Full disclosure: Tony Li signed off on my PhD scholarship funding when he was still at Cisco in 2011, however I have no other connection to him.

[kp]

One small issue is that this patch claims to move these files. That's clearly unintentional. Perhaps an artefact of how it was uploaded?

[olivier]

In D55558#1271209, @kp wrote:

One small issue is that this patch claims to move these files. That's clearly unintentional. Perhaps an artefact of how it was uploaded?

I’ve generated the diff with a git diff -U99 sys/netinet/ip_carp.c tests/sys/netinet/carp.sh > ~/vrrp.preempt.patch

[kp]

I haven't debugged it any depth (and won't be able to before early next week), but the new test case fails for me:

(kp@nut)  /usr/tests/sys/netinet % sudo kyua debug carp:vrrp_preempt                                                                                                               [14:00]
net.inet.carp.preempt: 0 -> 1
net.inet.carp.preempt: 0 -> 1
	vrrp: MASTER vrid 2 prio 10 interval 100
Files left in work directory after failure: created_interfaces.lst, created_jails.lst
ifconfig: interface epair0b does not exist
ifconfig: interface epair1b does not exist
carp:vrrp_preempt  ->  failed: preemption did not affect the second interface

[olivier]

In D55558#1271271, @kp wrote:

I haven't debugged it any depth (and won't be able to before early next week), but the new test case fails for me:

(kp@nut)  /usr/tests/sys/netinet % sudo kyua debug carp:vrrp_preempt                                                                                                               [14:00]
net.inet.carp.preempt: 0 -> 1
net.inet.carp.preempt: 0 -> 1
	vrrp: MASTER vrid 2 prio 10 interval 100
Files left in work directory after failure: created_interfaces.lst, created_jails.lst
ifconfig: interface epair0b does not exist
ifconfig: interface epair1b does not exist
carp:vrrp_preempt  ->  failed: preemption did not affect the second interface

But it works on both my side, and on my VM lab:

olivier@workstation:/usr/tests/sys/netinet $ sudo kyua debug carp:vrrp_preempt
net.inet.carp.preempt: 0 -> 1
net.inet.carp.preempt: 0 -> 1
        vrrp: MASTER vrid 2 prio 10 interval 100
ifconfig: interface epair0b does not exist
ifconfig: interface epair1b does not exist
carp:vrrp_preempt  ->  passed

[ae]

In D55558#1271208, @bms wrote:

Just my 2c: I think we should be discouraging users from implementing CARP at all.

But currently it is widely used by FreeBSD users. I personally have several hundreds of freebsd routers/firewalls that use CARP. And also we still have several patches to FreeBSD/bird/radvd to behave better.