Bug 210481 - [Carp] do not work correctly advertisement skew in the case of several vhid on a single physical interface
Summary: [Carp] do not work correctly advertisement skew in the case of several vhid o...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.3-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-net mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-23 03:15 UTC by Vladislav V. Prodan
Modified: 2018-08-02 13:42 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vladislav V. Prodan 2016-06-23 03:15:56 UTC
Hello.

I have two almost identical router server - bras1 and bras2.
FreeBSD 10.3-STABLE # 0: Wed Jun 22 07:33:13 EEST 2016

They planned to create more than a hundred vlan interface. As a backup and balancing traffic plan to use CARP.
It is also planned to use the PF NAT and pfsync for synchronization NAT sessions.
I do not work correctly advertisement skew in the case of several vhid on a single physical interface.

----------------------- bras1:

lagg0.1224: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9216
        inet XXX.YYY.ZZZ.228 netmask 0xfffffff0 broadcast XXX.YYY.ZZZ.239
        inet XXX.YYY.ZZZ.230 netmask 0xffffffff broadcast XXX.YYY.ZZZ.230 vhid 3
        carp: BACKUP vhid 3 advbase 1 advskew 50

		lagg1.800: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9198
        inet 10.8.0.2 netmask 0xffffffff broadcast 10.8.0.2 vhid 2
        inet 10.8.0.5 netmask 0xffffff00 broadcast 10.8.0.255
        inet 10.8.0.1 netmask 0xffffffff broadcast 10.8.0.1 vhid 1
        carp: BACKUP vhid 2 advbase 1 advskew 100
        carp: BACKUP vhid 1 advbase 1 advskew 50

/etc/rc.conf
...
pf_enable="YES"
pflog_enable="YES"
pflog_logfile="/var/log/pf.log"
ifconfig_pfsync0="mtu 9000 up"
pfsync_enable="YES"
pfsync_syncdev="lagg2"
pfsync_syncpeer="10.0.82.12"

ifconfig_lagg0_1224="inet XXX.YYY.ZZZ.228/28"
ifconfig_lagg0_1224_alias0="inet vhid 3 advskew 50 pass 52ca85c8ae9de7247a8eb7954da5dfe5 alias XXX.YYY.ZZZ.230/32"

ifconfig_lagg1_800_alias0="inet vhid 1 advskew  50 pass 7ac8c02055d2c4b5cc477b88c1f05733 alias 10.8.0.1/32"
ifconfig_lagg1_800_alias1="inet vhid 2 advskew 100 pass 6aeb230d8d55bfa3ab023cf0279ad424 alias 10.8.0.2/32"
ifconfig_lagg1_800_alias2="inet 10.8.0.5/24"
...

sysctl:
net.inet.carp.preempt: 1

		
----------------------- bras2:
		
lagg0.1224: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9216
        inet XXX.YYY.ZZZ.229 netmask 0xfffffff0 broadcast XXX.YYY.ZZZ.239
        inet XXX.YYY.ZZZ.230 netmask 0xffffffff broadcast XXX.YYY.ZZZ.230 vhid 3
        carp: MASTER vhid 3 advbase 1 advskew 100
lagg1.800: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9198
        inet 10.8.0.2 netmask 0xffffffff broadcast 10.8.0.2 vhid 2
        inet 10.8.0.6 netmask 0xffffff00 broadcast 10.8.0.255
        inet 10.8.0.1 netmask 0xffffffff broadcast 10.8.0.1 vhid 1
        carp: MASTER vhid 2 advbase 1 advskew 50
        carp: MASTER vhid 1 advbase 1 advskew 100
		
/etc/rc.conf
...
pf_enable="YES"
pflog_enable="YES"
pflog_logfile="/var/log/pf.log"
ifconfig_pfsync0="mtu 9000 up"
pfsync_enable="YES"
pfsync_syncdev="lagg2"
pfsync_syncpeer="10.0.82.11"
		
ifconfig_lagg0_1224="inet XXX.YYY.ZZZ.229/28"
ifconfig_lagg0_1224_alias0="inet vhid 3 advskew 100 pass 52ca85c8ae9de7247a8eb7954da5dfe5 alias XXX.YYY.ZZZ.230/32"

ifconfig_lagg1_800_alias0="inet vhid 1 advskew 100 pass 7ac8c02055d2c4b5cc477b88c1f05733 alias 10.8.0.1/32"
ifconfig_lagg1_800_alias1="inet vhid 2 advskew  50 pass 6aeb230d8d55bfa3ab023cf0279ad424 alias 10.8.0.2/32"
ifconfig_lagg1_800_alias2="inet 10.8.0.6/24"
...

sysctl:
net.inet.carp.preempt: 1


It r297791, but upgrade to r302074 not help :(
Change net.inet.carp.preempt from 0 to 1 or vice versa, too, especially not affect the behavior of the CARP.

As a temporary measure to help the team, introduced to bras1:
ifconfig lagg1.800 vhid 2 state backup
ifconfig lagg0.1224 vhid 3 state master

At the same time in the next pair of servers (r273154) advskew works correctly.

r273154 FreeBSD 10.1-PRERELEASE #0: Thu Oct 16 03:58:25 EEST 2014

After reboot:
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 00:25:90:00:58:fd
        inet 10.10.12.12 netmask 0xffffff00 broadcast 10.10.12.255
        inet 10.10.13.11 netmask 0xffffff00 broadcast 10.10.13.255 vhid 1
        inet 10.10.13.12 netmask 0xffffff00 broadcast 10.10.13.255 vhid 2
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        carp: BACKUP vhid 1 advbase 1 advskew 100
        carp: MASTER vhid 2 advbase 1 advskew 50

/etc/rc.conf
...
ifconfig_em1="inet 10.10.12.12/24 mtu 9000"
ifconfig_em1_alias0="vhid 1 advskew 100 pass 10bdbe7b138e4e69efa0e72d108a13a3 alias 10.10.13.11/24"
ifconfig_em1_alias1="vhid 2 advskew  50 pass 6aeb230d8d55bfa3ab023cf0279ad424 alias 10.10.13.12/24"
...		

sysctl:
net.inet.carp.preempt: 1