Bug 214746

Summary:	Carp ipv6 duplicate address detection
Product:	Base System	Reporter:	Athanasios Douitsis <aduitsis>
Component:	kern	Assignee:	freebsd-net (Nobody) <net>
Status:	Open ---
Severity:	Affects Some People	CC:	ae, dpetrov67, nicolas, stb, thomas
Priority:	---	Keywords:	needs-qa
Version:	10.3-RELEASE
Hardware:	amd64
OS:	Any

Description Athanasios Douitsis 2016-11-22 16:18:49 UTC

Somewhat difficult to ascertain whether this has been previously reported, apologies if this is a duplicate bug.

Trying to use carp and IPv6 on a 10.3-RELEASE. Two machines, each one with its own IPv6 address and one common IPv6. Fairly simple.

When a machine boots (the other one is obviously MASTER and has acquired the common IPv6 address), complaints about duplicate address detection of the common IPv6 are logged by the booting machine's kernel.

kernel: vmx0: DAD detected duplicate IPv6 address <commonipv6address>: NS in/out/loopback=0/1/0, NA in=1
kernel: vmx0: DAD complete for <commonipv6address> - duplicate found

As a result, the machine that has just booted has the duplicated flag for that address in ifconfig and services that want to bind to that common address fail to start. In other words, the BACKUP cannot bind to the common address.

Trying to setup the common address and carp by hand using ifconfig initially fails with:

#ifconfig vmx0 inet6 <commonipv6address>/64 vhid 34 pass <my_pass>
ifconfig: ioctl (SIOCAIFADDR): No such file or directory

What's curious, second time that command is issued, no errors are printed and the address is assigned to the interface.

Using the no_dad ifconfig flag seems to solve the problem, but I suspect that this workaround is not extremely good as it disables DAD.

Comment 1 Kubilay Kocak freebsd_committer

2016-12-03 02:42:51 UTC

@Athanasios Could you please attach a file containing the non-working and working configurations (ie: /etc/rc.conf, if specified there). 

Please also include the exact FreeBSD version (uname -a) output.

Comment 2 Athanasios Douitsis 2016-12-03 09:53:00 UTC

(In reply to Kubilay Kocak from comment #1)

Running FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:49:24 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64 on both nodes.

This works (dead:beef instead of my real prefix):

ifconfig_vmx0_ipv6="inet6 accept_rtadv dead:beef::60/64"
ifconfig_vmx0_alias1="inet6 dead:beef::95/64 no_dad vhid 34 pass bar”

dead:beef::60 is node0 and dead:beef::61 is node1. Both have dead:beef::95/64 as the common carp IPv6 address. Vhid 34 is randomly chosen. 

For IPv4, I'm using a different vhid and pass, just to be on the safe side.

Removing the no_dad from ifconfig_vmx0_alias1 and rebooting the BACKUP node causes the problem to appear. Backup node will compain about DAD upon reboot, ifconfig will say dead:beef::95/64 duplicated, so it can't really be used. 

Afterwards, trying to manualy issue 

ifconfig inet6 dead:beef::95/64 vhid 34 pass bar

will fail the first time, second time it will go through.

Comment 3 Nicolas 2022-01-05 21:01:25 UTC

Hello,

FreeBSD 12.3-RELEASE

The issue is still exist but in a strange way, only the first carp interface have duplicated flag, other interfaces didn't have it (of course interfaces are configured in same way)

host1-master# ifconfig vlan6
vlan6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether aa:aa:aa:aa:aa:aa
	inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255
	inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 vhid 6
	inet6 fe80::aaaa:aaaa:aaaa:aaaa%vlan6 prefixlen 64 scopeid 0xf
	inet6 aaaa:bbbb:cccc:1::2 prefixlen 64
	inet6 aaaa:bbbb:cccc:1::1 prefixlen 64 duplicated vhid 6
	groups: vlan
	carp: MASTER vhid 6 advbase 1 advskew 0
	vlan: 106 vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

host1-master# ifconfig vlan7
vlan7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether aa:aa:aa:aa:aa:aa
	inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 vhid 7
	inet6 fe80::aaaa:aaaa:aaaa:aaaa%vlan7 prefixlen 64 scopeid 0x10
	inet6 aaaa:bbbb:cccc:2::2 prefixlen 64
	inet6 aaaa:bbbb:cccc:2::1 prefixlen 64 vhid 7
	groups: vlan
	carp: MASTER vhid 7 advbase 1 advskew 0
	vlan: 107 vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect

host2-slave# ifconfig vlan6
vlan6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether bb:bb:bb:bb:bb:bb
	inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
	inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 vhid 6
	inet6 fe80::bbbb:bbbb:bbbb:bbbb%vlan6 prefixlen 64 scopeid 0xf
	inet6 aaaa:bbbb:cccc:1::3 prefixlen 64
	inet6 aaaa:bbbb:cccc:1::1 prefixlen 64 vhid 6
	groups: vlan
	carp: BACKUP vhid 6 advbase 1 advskew 100
	vlan: 106 vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

host2-slave# ifconfig vlan7
vlan7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether bb:bb:bb:bb:bb:bb
	inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 vhid 7
	inet6 fe80::bbbb:bbbb:bbbb:bbbb%vlan7 prefixlen 64 scopeid 0x10
	inet6 aaaa:bbbb:cccc:2::3 prefixlen 64
	inet6 aaaa:bbbb:cccc:2::1 prefixlen 64 vhid 7
	groups: vlan
	carp: BACKUP vhid 7 advbase 1 advskew 100
	vlan: 107 vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
# 

As suggested by Athanasios, entering manually again the address make duplicated flag disappear and so IPv6 address working.

host1-master# ifconfig vlan6 inet6 aaaa:bbbb:cccc:1::1 prefixlen 64 vhid6

host1-master# ifconfig vlan6
vlan6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether aa:aa:aa:aa:aa:aa
	inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255
	inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 vhid 6
	inet6 fe80::aaaa:aaaa:aaaa:aaaa%vlan6 prefixlen 64 scopeid 0xf
	inet6 aaaa:bbbb:cccc:1::2 prefixlen 64
	inet6 aaaa:bbbb:cccc:1::1 prefixlen 64 vhid 6
	groups: vlan
	carp: MASTER vhid 6 advbase 1 advskew 0
	vlan: 106 vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

host1-master#

Comment 4 Thomas Steen Rasmussen / Tykling 2022-01-05 21:04:24 UTC

I saw this just the other day! Thought it was me going nuts. Will keep an eye out for it and note the circumstances next time, but it was on 12.3-STABLE

Comment 5 Stefan Bethke 2023-07-25 06:46:36 UTC

I just ran into this myself, 12.4-stable; I have carp for ipv4, plus a custom script that configures the v6 VIP manually. I don't think I've seen this hehavior before, where the failover has this issue.

I'll probably adjust my script to either wait a second or two, or watch out for the duplicate flag and simply try again.

I couldn't find much documentation on the duplicate flag. As far as I can tell, it sticks and disables that address permanently. I would have expected that there would be a retry mechanism of some sort.

Comment 6 Andrey V. Elsukov freebsd_committer

2023-07-25 10:06:05 UTC

You can use no_dad flag when you create vlan interface. We usually configure all addresses manually on carp interfaces and also use -auto_linklocal flag:

# vlan416
carp_interfaces="${carp_interfaces} vlan416"
ifconfig_vlan416="vlan 416 vlandev ${vlanif} mtu 1500"
ifconfig_vlan416_ipv6="inet6 -auto_linklocal no_dad"
ifconfig_vlan416_alias1="inet A.B.C.D/28 vhid N ${carp_args}"
ifconfig_vlan416_alias2="inet6 XX:XX:XX:XX::1/64 vhid M ${carp_args}"
ifconfig_vlan416_alias3="inet6 fe80::1/64 vhid M ${carp_args}"