Bug 202510

Summary: [CARP] advertisements sourced from CARP IP cause double MASTER situations
Product: Base System Reporter: Damien Fleuriot <dam>
Component: kernAssignee: freebsd-net (Nobody) <net>
Status: Open ---    
Severity: Affects Many People CC: girgen, glebius, koobs
Priority: --- Keywords: easy, needs-qa, patch
Version: 10.0-STABLEFlags: koobs: mfc-stable11?
koobs: mfc-stable10?
koobs: mfc-stable9?
Hardware: Any   
OS: Any   
Attachments:
Description Flags
network.subr IP alias order patch none

Description Damien Fleuriot 2015-08-20 10:07:30 UTC
We're running Host A 10.2-BETA and Host B 10.2-PRERELEASE.

Host A is MASTER and Host B is supposed to be BACKUP.

When CARPs are configured from rc.conf on Host B, I get into a MASTER-MASTER situation.
When CARPs are configured manually on Host B, I get the normal behaviour.

The bug is 100% reproductible.



We've managed to narrow it down to the syntax used in /etc/rc.conf to configure the physical IP for the interface :


When using the following syntax, the physical IP address is configured AFTER the CARPs on the interface, which results in the CARP advertisements being sourced from the CARP IP, triggering the double MASTER situation :
ipv4_addrs_int="1.2.3.4/24"
ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20"

When using either of the following syntaxes, the physical IP address is configured BEFORE the CARPs, which results in the CARP advertisements being sourced from the physical IP and restoring normal functionality :
ifconfig_int="inet 1.2.3.4/24"
ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20"
OR
ifconfig_int_alias0="1.2.3.4/24"
ifconfig_int_alias1="1.2.3.6/32 vhid 1 pass test advskew 20"
Comment 1 Damien Fleuriot 2015-08-24 09:05:57 UTC
As a precision, the problem stems more from how IPs are assigned from rc , than from CARP itself.

Either the old ipv4_addrs_<interface> syntax should be deprecated, or it should be moved higher up so that it's enacted before ifconfig_<interface>_aliasN.
Comment 2 Damien Fleuriot 2015-08-24 14:16:11 UTC
Created attachment 160307 [details]
network.subr IP alias order patch

Ensures "ipv4_addrs_<interface>" addresses are set up before "ifconfig_<interface>_aliasN" addresses.

This makes sure if we're going to use CARPs *in addition to physical IPs*, the advertisements will be sent from the first configured IP.
Comment 3 Damien Fleuriot 2015-08-24 14:18:06 UTC
Patch attached, changes rc's behaviour so that "ipv4_addrs" addresses are set up on interfaces before "ifconfig_aliasN" addresses.

Seeing this changes the system's behaviour, this may require consensus before approval.
Comment 4 Palle Girgensohn freebsd_committer freebsd_triage 2016-11-29 22:14:16 UTC
(In reply to dam from comment #3)

Any progress with getting consensus?
Comment 5 Damien Fleuriot 2016-11-30 08:50:57 UTC
(In reply to Palle Girgensohn from comment #4)


Hello Palle,


Ah there may have been a small misunderstanding here.

My statement was more meant as a "the patch is made by a novice to RC and network.subr, likely needs review/consensus from devs".

It didn't mean I was going to seek consensus -from the MLs for example- .

I thought someone from net@ would take the ticket, review the patch and ensure it doesn't introduce any regression.
Comment 6 Palle Girgensohn freebsd_committer freebsd_triage 2016-11-30 08:56:35 UTC
Ah, OK. Good. So, hopefully someone will nurture this patch then. Any takers?
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-30 10:13:11 UTC
(In reply to dam from comment #3)

Presumably testing shows that both syntaxes now result in the correct (expected) behaviour. Can you confirm?
Comment 8 Damien Fleuriot 2016-12-01 09:09:56 UTC
(In reply to Kubilay Kocak from comment #7)

svnup'ing 10-STABLE now, applying patch, and getting back to you.

I'm afraid our company has neither 9.x nor 11.x available for testing against these branches though.

Anyone in a position to test against these ?
Comment 9 Damien Fleuriot 2016-12-01 16:11:16 UTC
(In reply to Kubilay Kocak from comment #7)


Confirming behaviour on 10-STABLE (r309356) with my provided patch.




patch
==
root@bsd10:/usr/src/etc # patch < /tmp/network.subr.patch
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- network.subr.orig	2015-08-24 14:10:15.188579000 +0000
|+++ network.subr.patched	2015-08-24 14:10:22.298347000 +0000
--------------------------
File to patch: network.subr
Patching file network.subr using Plan A...
Hunk #1 succeeded at 1091.
Hunk #2 succeeded at 1144.
done




rc.conf
==
ifconfig_vtnet0="inet 46.[snip].253/25 -rxcsum -txcsum -tso -vlanhwtso -vlanhwcsum -vlanhwtag up"
ipv4_addrs_vtnet0="1.1.1.1/32"
ifconfig_vtnet0_alias0="1.1.1.2/32"
ifconfig_vtnet0_alias1="vhid 101 pass test advskew 50 1.1.1.3/32"




ifconfig
==
root@bsd10:~ # ifconfig
vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=6804a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,LRO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether f2:53:d5:fc:49:8a
	inet 46.[snip].253 netmask 0xffffff80 broadcast 46.182.42.255 
	inet 1.1.1.1 netmask 0xffffffff broadcast 1.1.1.1 
	inet 1.1.1.2 netmask 0xffffffff broadcast 1.1.1.2 
	inet 1.1.1.3 netmask 0xffffffff broadcast 1.1.1.3 vhid 101 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T <full-duplex>
	status: active
	carp: MASTER vhid 101 advbase 1 advskew 50




tcpdump
==
root@bsd10:~ # tcpdump -ni vtnet0 vrrp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:04:27.942147 IP 46.[snip].253 > 224.0.0.18: VRRPv2, Advertisement, vrid 101, prio 50, authtype none, intvl 1s, length 36
16:04:29.156181 IP 46.[snip].253 > 224.0.0.18: VRRPv2, Advertisement, vrid 101, prio 50, authtype none, intvl 1s, length 36
16:04:30.358144 IP 46.[snip].253 > 224.0.0.18: VRRPv2, Advertisement, vrid 101, prio 50, authtype none, intvl 1s, length 36

     ^---- sourced with physical IP despite the use of ipv4_addrs_vtnet0



Can anyone confirm 9 and 11 branches ?