when an inet6 address is configured on a carp interface, the master works fine, but the backup is constantly switching between master and slave. It worked fine on FreeBSD 6.0, the problem appeared after upgrading from 6.0 to 6.1 While looking with tcpdump, I can't see IPv6 carp advertisement on FreeBSD 6.1, whereas I can see it with FreeBSD 6.0 How-To-Repeat: two box with FreeBSD 6.1 in failover with carp, configure an IPv6 address on the carp interface and you can see the backup switching from master to slave in loop.
I found the culprit to be the commit dated "Fri Nov 4 20:26:14 2005 UTC" (cf 1.27.2.3 revision of ip_carp.c for example). Before this commit, everything works fine, and I can watch CARP advertising packets going out with IPv6 *and* IPv4 from the master. After this commit, only IPv4 packets are sent. In order to test, I built the following test-bed: 2 servers, one being the CARP master (s1) and the second being the slave (s2). For each tested version, I executed the following commands on the master: ifconfig carp0 create ifconfig carp0 vhid 219 pass carptest 130.79.201.235/24 ifconfig carp0 inet6 2001:660:2402:1001::235/64 and the following commands on the slave: ifconfig carp0 create ifconfig carp0 vhid 219 advskew 100 pass carptest 130.79.201.235/24 ifconfig carp0 inet6 2001:660:2402:1001::235/64 Before the commit: s1# tcpdump -eni xl0 proto carp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on xl0, link-type EN10MB (Ethernet), capture size 96 bytes 11:41:01.326923 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 11:41:01.326984 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36 11:41:02.327759 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 11:41:02.327805 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36 11:41:03.328604 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 11:41:03.328650 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36 11:41:04.329447 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 11:41:04.329485 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36 ^C 8 packets captured 35 packets received by filter 0 packets dropped by kernel After the commit, the slave continuously attempts to become master, since IPv6 CARP advertisements are not sent by the master: s1# tcpdump -eni xl0 proto carp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on xl0, link-type EN10MB (Ethernet), capture size 96 bytes 15:05:46.976380 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 15:05:47.977221 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 15:05:48.978064 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 15:05:49.978910 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 15:05:50.979757 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36 ^C 5 packets captured 42 packets received by filter 0 packets dropped by kernel One more strange thing: when I add WITNESS and INVARIANTS options in the kernel (before the commit, I didn't test after it), the kernel panics when I add an IPv6 address to the CARP interface: # ifconfig carp0 inet6 2001:660:2402:1001::235/64 panic: _mtx_lock_sleep: recursed on non-recursive mutex carp_if @/usr/src/sys/netinet/ip_carp.c:1209 KDB: enter: panic [thread pid 28 tid 100024 ] Stopped at kdb_enter+0x2b: nop db> trace Tracing pid 28 tid 100024 td 0xc1522600 kdb_enter(c0894127) at kdb_enter+0x2b panic(c0893618,c08a1832,c08a152c,4b9,c1522678) at panic+0xbb _mtx_lock_sleep(c18d14d0,c1522600,0,c08a152c,4b9) at _mtx_lock_sleep+0x3a _mtx_lock_flags(c18d14d0,0,c08a152c,4b9,c1730aa8) at _mtx_lock_flags+0x84 carp_macmatch6(c18d14c0,c1725700,c17306a8,c09d9460,0) at carp_macmatch6+0x27 nd6_na_output(c1624400,c0903994,c17306a8,20,1) at nd6_na_output+0x23e carp_send_na(c195c200,c195c200,c195c200,c195c200,2) at carp_send_na+0x36 carp_master_down_locked(c195c200,c18d14d0,0,c08a152c,4f4) at carp_master_down_locked+0x72 carp_master_down(c195c200) at carp_master_down+0x28 softclock(0) at softclock+0x211 ithread_loop(c14f0400,cc9d5d38,c14f0400,c0654f50,0) at ithread_loop+0x144 fork_exit(c0654f50,c14f0400,cc9d5d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xcc9d5d6c, ebp = 0 ---
I can confirm this worked in 6.0-R and is broken in 6.1-R, and I can confirm it was the commit by ume@freebsd.org mentioned in the past log entries of this PR that broke it: http://docs.FreeBSD.org/cgi/mid.cgi?200511042026.jA4KQGX9038319 In my case, I don't see the backup constantly switching, instead I see both the master and backup constantly acting as master, but only on the IPv6 interface - the IPv4 interface works as expected. This difference could be due to te fact that I have the IPv4 interface on this machine also under carp's control, although I do have sysctl net.inet.carp.preempt=1, which in theory should mean that all interfaces track each other's state. Gavin
Hi, >>>>> On Thu, 27 Jul 2006 17:03:40 +0100 (BST) >>>>> Gavin Atkinson <gavin.atkinson@ury.york.ac.uk> said: gavin.atkinson> I can confirm this worked in 6.0-R and is broken in 6.1-R, and I can gavin.atkinson> confirm it was the commit by ume@freebsd.org mentioned in the past log gavin.atkinson> entries of this PR that broke it: gavin.atkinson> http://docs.FreeBSD.org/cgi/mid.cgi?200511042026.jA4KQGX9038319 ipcarp.c:1.27.2.3 changed to just use the scope API rather than hard coded scope things, and itself seems correct. However, it might change some semantics. I don't have an environment for testing CARP. Please backout 1.27.2.3 by applying the following patch, and let me the result. Index: sys/netinet/ip_carp.c diff -u -p sys/netinet/ip_carp.c.orig sys/netinet/ip_carp.c --- sys/netinet/ip_carp.c.orig Mon Dec 26 06:59:20 2005 +++ sys/netinet/ip_carp.c Mon Jul 10 16:50:22 2006 @@ -269,7 +269,8 @@ carp_hmac_prepare(struct carp_softc *sc) TAILQ_FOREACH(ifa, &SC2IFP(sc)->if_addrlist, ifa_list) { if (ifa->ifa_addr->sa_family == AF_INET6) { in6 = ifatoia6(ifa)->ia_addr.sin6_addr; - in6_clearscope(&in6); + if (IN6_IS_ADDR_LINKLOCAL(&in6)) + in6.s6_addr16[1] = 0; SHA1Update(&sc->sc_sha1, (void *)&in6, sizeof(in6)); } } @@ -1543,7 +1544,7 @@ carp_set_addr6(struct carp_softc *sc, st struct in6_ifaddr *ia, *ia_if; struct ip6_moptions *im6o = &sc->sc_im6o; struct in6_multi_mship *imm; - struct in6_addr in6; + struct sockaddr_in6 addr; int own, error; if (IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr)) { @@ -1592,25 +1593,25 @@ carp_set_addr6(struct carp_softc *sc, st im6o->im6o_multicast_ifp = ifp; /* join CARP multicast address */ - bzero(&in6, sizeof(in6)); - in6.s6_addr16[0] = htons(0xff02); - in6.s6_addr8[15] = 0x12; - if (in6_setscope(&in6, ifp, NULL) != 0) - goto cleanup; - if ((imm = in6_joingroup(ifp, &in6, &error, 0)) == NULL) + bzero(&addr, sizeof(addr)); + addr.sin6_family = AF_INET6; + addr.sin6_len = sizeof(addr); + addr.sin6_addr.s6_addr16[0] = htons(0xff02); + addr.sin6_addr.s6_addr16[1] = htons(ifp->if_index); + addr.sin6_addr.s6_addr8[15] = 0x12; + if ((imm = in6_joingroup(ifp, &addr.sin6_addr, &error, 0)) == NULL) goto cleanup; LIST_INSERT_HEAD(&im6o->im6o_memberships, imm, i6mm_chain); /* join solicited multicast address */ - bzero(&in6, sizeof(in6)); - in6.s6_addr16[0] = htons(0xff02); - in6.s6_addr32[1] = 0; - in6.s6_addr32[2] = htonl(1); - in6.s6_addr32[3] = sin6->sin6_addr.s6_addr32[3]; - in6.s6_addr8[12] = 0xff; - if (in6_setscope(&in6, ifp, NULL) != 0) - goto cleanup; - if ((imm = in6_joingroup(ifp, &in6, &error, 0)) == NULL) + bzero(&addr.sin6_addr, sizeof(addr.sin6_addr)); + addr.sin6_addr.s6_addr16[0] = htons(0xff02); + addr.sin6_addr.s6_addr16[1] = htons(ifp->if_index); + addr.sin6_addr.s6_addr32[1] = 0; + addr.sin6_addr.s6_addr32[2] = htonl(1); + addr.sin6_addr.s6_addr32[3] = sin6->sin6_addr.s6_addr32[3]; + addr.sin6_addr.s6_addr8[12] = 0xff; + if ((imm = in6_joingroup(ifp, &addr.sin6_addr, &error, 0)) == NULL) goto cleanup; LIST_INSERT_HEAD(&im6o->im6o_memberships, imm, i6mm_chain); } Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@{,jp.}FreeBSD.org http://www.imasy.org/~ume/
On Fri, 28 Jul 2006, Hajimu UMEMOTO wrote: > ipcarp.c:1.27.2.3 changed to just use the scope API rather than hard > coded scope things, and itself seems correct. However, it might > change some semantics. > I don't have an environment for testing CARP. Please backout 1.27.2.3 > by applying the following patch, and let me the result. [patch deleted] Backing just that part of the larger commit out (and changing it so it compiled - in6_joingroup only takes three arguments) made no difference to the problems seen. I'm happy to test any other patches or add printf's to any bits of code you would like me to try, but please be aware that I will only be online occassionally over the next three days. Gavin
Responsible Changed From-To: freebsd-bugs->ume-bugs Hajimu-san, please handle the regression you have introduced. If you have any questions about CARP internals, I can answer your questions. Sorry, I can't handle this regression myself, since I'm too lame in IPv6.
Responsible Changed From-To: ume-bugs->ume Hajimu-san, please handle the regression you have introduced. If you have any questions about CARP internals, I can answer your questions. Sorry, I can't handle this regression myself, since I'm too lame in IPv6.
Hi, Sorry for my delayed response. >>>>> On Thu, 27 Jul 2006 18:46:06 +0100 (BST) >>>>> Gavin Atkinson <gavin.atkinson@ury.york.ac.uk> said: gavin> I'm happy to test any other patches or add printf's to any bits of code gavin> you would like me to try, but please be aware that I will only be online gavin> occassionally over the next three days. Could you try this patch? Index: sys/netinet/ip_carp.c diff -u -p sys/netinet/ip_carp.c.orig sys/netinet/ip_carp.c --- sys/netinet/ip_carp.c.orig Sat Aug 19 18:56:27 2006 +++ sys/netinet/ip_carp.c Wed Aug 30 01:12:44 2006 @@ -977,6 +977,12 @@ carp_send_ad_locked(struct carp_softc *s ch_ptr->carp_cksum = carp_cksum(m, len - sizeof(*ip6)); m->m_data -= sizeof(*ip6); + if (in6_setscope(&ip6->ip6_dst, SC2IFP(sc), NULL) != 0) { + m_freem(m); + SC2IFP(sc)->if_oerrors++; + return; + } + getmicrotime(&SC2IFP(sc)->if_lastchange); SC2IFP(sc)->if_opackets++; SC2IFP(sc)->if_obytes += len; @@ -1042,7 +1048,10 @@ carp_send_na(struct carp_softc *sc) { struct ifaddr *ifa; struct in6_addr *in6; - static struct in6_addr mcast = IN6ADDR_LINKLOCAL_ALLNODES_INIT; + struct in6_addr mcast = IN6ADDR_LINKLOCAL_ALLNODES_INIT; + + if (in6_setscope(&mcast, sc->sc_carpdev, NULL) != 0) + return; TAILQ_FOREACH(ifa, &SC2IFP(sc)->if_addrlist, ifa_list) { Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@{,jp.}FreeBSD.org http://www.imasy.org/~ume/
On Wed, 2006-08-30 at 01:25 +0900, Hajimu UMEMOTO wrote: > Hi, > > Sorry for my delayed response. No problem at all. > Could you try this patch? I'm afraid it makes no difference. If it will help, I can probably provide access to two test machines which exhibit the problem and which you would be free to play with, or if you have two spare machines I can give you two rc.conf files which will demonstrate the problem at your end - you don't need any IPv6 infrastructure to see the issue, just two machines on the same network. Gavin
Hajimu UMEMOTO wrote: > Hi, Hi, > > Sorry for my delayed response. > >>>>>> On Thu, 27 Jul 2006 18:46:06 +0100 (BST) >>>>>> Gavin Atkinson <gavin.atkinson@ury.york.ac.uk> said: > > gavin> I'm happy to test any other patches or add printf's to any bits of code > gavin> you would like me to try, but please be aware that I will only be online > gavin> occassionally over the next three days. > > Could you try this patch? > ... I tested your patch against today 6-STABLE but nothing better (no IPv6 CARP advertising packets going out). One more strange thing : when I add WITNESS and INVARIANTS options in the kernel, the kernel panic when I add an IPv6 address to the CARP interface : # panic: _mtx_lock_sleep: recursed on non-recursive mutex carp_if @ /usr/src/sys/netinet/ip_carp.c:1200 KDB: enter: panic [thread pid 13 tid 100003 ] Stopped at kdb_enter+0x2b: nop db> trace Tracing pid 13 tid 100003 td 0xc2159a80 kdb_enter(c08ba6e7) at kdb_enter+0x2b panic(c08b9b7e,c08c8803,c08c84fd,4b0,c2159af8,...) at panic+0xbb _mtx_lock_sleep(c260add0,c2159a80,0,c08c84fd,4b0) at _mtx_lock_sleep+0x3a _mtx_lock_flags(c260add0,0,c08c84fd,4b0,cbfa4bdc,...) at _mtx_lock_flags+0xa1 carp_macmatch6(c260adc0,c227eb00,c23818a8,c08cd05d,0,...) at carp_macmatch6+0x27 nd6_na_output(c22c9400,cbfa4c48,c23818a8,20,1,...) at nd6_na_output+0x2c3 carp_send_na(c2381e00,c2381e00,c2381e00,c2381e00,2,...) at carp_send_na+0x69 carp_master_down_locked(c2381e00,c260add0,0,c08c84fd,4eb,...) at carp_master_down_locked+0x72 carp_master_down(c2381e00) at carp_master_down+0x28 softclock(0) at softclock+0x211 ithread_execute_handlers(c2158624,c2156480) at ithread_execute_handlers+0xe6 ithread_loop(c213c6f0,cbfa4d38,c213c6f0,c0670fd4,0,...) at ithread_loop+0x66 fork_exit(c0670fd4,c213c6f0,cbfa4d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xcbfa4d6c, ebp = 0 --- -- Philippe Pegon
I am still able to provide remote root access to two test machines displaying the problem, and a third to monitor network traffic on, if required. Gavin
Responsible Changed From-To: ume->bz friendly takeover;) I have spent the last 2 hours analyzing this and will hopefully be able to come up with a (tested) patch later today or tomorrow in time for 6.2-${next}.
Can you try this patch (also added one for ip_fw2.c to not make it complain like: IPFW2: IPV6 - Unknown Extension Header(112), ext_hd=0). You can also (temporary) fetch it from: http://sources.zabbadoz.net/freebsd/patchset/20061005-01-carp-v6-scope-ipfw.diff The cause seems to be that in ip6_output a check was added with the previously mentioned commit that (correctly) failed: [ip6_output:xxx] sa6_recoverscope dst ff02::0012, zone=1 != dst_sa.sin6_scope_id=0 and you would see netstat -s -p ip6 | grep scope 0 packets that violated scope rules being incremented for every carp advertisement being sent out with an ipv6 address. Index: sys/netinet/ip_carp.c =================================================================== RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_carp.c,v retrieving revision 1.27.2.8 diff -u -p -r1.27.2.8 ip_carp.c --- sys/netinet/ip_carp.c 25 Sep 2006 13:01:59 -0000 1.27.2.8 +++ sys/netinet/ip_carp.c 5 Oct 2006 21:47:33 -0000 @@ -965,9 +965,12 @@ carp_send_ad_locked(struct carp_softc *s sizeof(struct in6_addr)); /* set the multicast destination */ - ip6->ip6_dst.s6_addr8[0] = 0xff; - ip6->ip6_dst.s6_addr8[1] = 0x02; + ip6->ip6_dst.s6_addr16[0] = htons(0xff02); ip6->ip6_dst.s6_addr8[15] = 0x12; + if (in6_setscope(&ip6->ip6_dst, sc->sc_carpdev, NULL) != 0) { + CARP_LOG("%s: in6_setscope failed\n", __func__); + return; + } ch_ptr = (struct carp_header *)(&ip6[1]); bcopy(&ch, ch_ptr, sizeof(ch)); Index: sys/netinet/ip_fw2.c =================================================================== RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_fw2.c,v retrieving revision 1.106.2.20 diff -u -p -r1.106.2.20 ip_fw2.c --- sys/netinet/ip_fw2.c 20 Sep 2006 19:15:03 -0000 1.106.2.20 +++ sys/netinet/ip_fw2.c 5 Oct 2006 21:47:33 -0000 @@ -73,6 +73,7 @@ #include <netinet/ip_fw.h> #include <netinet/ip_divert.h> #include <netinet/ip_dummynet.h> +#include <netinet/ip_carp.h> #include <netinet/pim.h> #include <netinet/tcp.h> #include <netinet/tcp_timer.h> @@ -2315,6 +2316,16 @@ do { \ PULLUP_TO(hlen, ulp, struct pim); break; + case IPPROTO_CARP: + PULLUP_TO(hlen, ulp, struct carp_header); + if (((struct carp_header *)ulp)->carp_version != + CARP_VERSION) + return (IP_FW_DENY); + if (((struct carp_header *)ulp)->carp_type != + CARP_ADVERTISEMENT) + return (IP_FW_DENY); + break; + case IPPROTO_IPV6: /* RFC 2893 */ PULLUP_TO(hlen, ulp, struct ip6_hdr); break; -- Bjoern A. Zeeb bzeeb at Zabbadoz dot NeT
On Fri, 6 Oct 2006, SUZUKI Shinsuke wrote: > Could you please try the attached patch (this is for RELENG_6, but the > same patch would probably work for -CURRENT)? *grrr* why was that information not in gnats yesterday evening? Someone else owes me some hours of sleep;-)) > I can reproduce the phenomena and it seems like my patch fixes it. yupp; should we use sc->sc_carpdev or sc->sc_ia6->ia_ifp? It's just a whitespace question. > Here's my analysis: > The destination address (ff02::12) of IPv6 CARP packet does not > contain a scope information. In such case, IPv6 stack does now allow > the advertisement of CARP packet, because of an inconsistent scope > boundary information, which was introduced in the patch you mentioned > above. (i.e. the destination address must have a link-local scope, > but there is no scope information at all) Same conclusion that I came to last night. See the PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=98622 > Index: ip_carp.c > =================================================================== > RCS file: /home/ncvs/src/sys/netinet/ip_carp.c,v > retrieving revision 1.27.2.8 > diff -u -p -r1.27.2.8 ip_carp.c > --- ip_carp.c 25 Sep 2006 13:01:59 -0000 1.27.2.8 > +++ ip_carp.c 6 Oct 2006 03:52:03 -0000 > @@ -968,6 +968,7 @@ carp_send_ad_locked(struct carp_softc *s > ip6->ip6_dst.s6_addr8[0] = 0xff; > ip6->ip6_dst.s6_addr8[1] = 0x02; > ip6->ip6_dst.s6_addr8[15] = 0x12; > + in6_setscope(&ip6->ip6_dst, sc->sc_ia6->ia_ifp, NULL); > > ch_ptr = (struct carp_header *)(&ip6[1]); > bcopy(&ch, ch_ptr, sizeof(ch)); -- Bjoern A. Zeeb bzeeb at Zabbadoz dot NeT
After discussion with glebius I updated the patch to a) plug an mbuf leak (yeah it was late last night;) b) also increment the carp counters for failures so it'll know sending out the packet didn't work an become BACKUP. We decided it would be better to log the error here instead of failing in ip6_output. Index: sys/netinet/ip_carp.c =================================================================== RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_carp.c,v retrieving revision 1.27.2.8 diff -u -p -r1.27.2.8 ip_carp.c --- sys/netinet/ip_carp.c 25 Sep 2006 13:01:59 -0000 1.27.2.8 +++ sys/netinet/ip_carp.c 6 Oct 2006 08:48:30 -0000 @@ -965,9 +965,14 @@ carp_send_ad_locked(struct carp_softc *s sizeof(struct in6_addr)); /* set the multicast destination */ - ip6->ip6_dst.s6_addr8[0] = 0xff; - ip6->ip6_dst.s6_addr8[1] = 0x02; + ip6->ip6_dst.s6_addr16[0] = htons(0xff02); ip6->ip6_dst.s6_addr8[15] = 0x12; + if (in6_setscope(&ip6->ip6_dst, sc->sc_carpdev, NULL) != 0) { + SC2IFP(sc)->if_oerrors++; + m_freem(m); + CARP_LOG("%s: in6_setscope failed\n", __func__); + return; + } ch_ptr = (struct carp_header *)(&ip6[1]); bcopy(&ch, ch_ptr, sizeof(ch)); Index: sys/netinet/ip_fw2.c =================================================================== RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_fw2.c,v retrieving revision 1.106.2.20 diff -u -p -r1.106.2.20 ip_fw2.c --- sys/netinet/ip_fw2.c 20 Sep 2006 19:15:03 -0000 1.106.2.20 +++ sys/netinet/ip_fw2.c 5 Oct 2006 21:47:33 -0000 @@ -73,6 +73,7 @@ #include <netinet/ip_fw.h> #include <netinet/ip_divert.h> #include <netinet/ip_dummynet.h> +#include <netinet/ip_carp.h> #include <netinet/pim.h> #include <netinet/tcp.h> #include <netinet/tcp_timer.h> @@ -2315,6 +2316,16 @@ do { \ PULLUP_TO(hlen, ulp, struct pim); break; + case IPPROTO_CARP: + PULLUP_TO(hlen, ulp, struct carp_header); + if (((struct carp_header *)ulp)->carp_version != + CARP_VERSION) + return (IP_FW_DENY); + if (((struct carp_header *)ulp)->carp_type != + CARP_ADVERTISEMENT) + return (IP_FW_DENY); + break; + case IPPROTO_IPV6: /* RFC 2893 */ PULLUP_TO(hlen, ulp, struct ip6_hdr); break; -- Bjoern A. Zeeb bzeeb at Zabbadoz dot NeT
Bjoern A. Zeeb wrote: > After discussion with glebius I updated the patch to > a) plug an mbuf leak (yeah it was late last night;) > b) also increment the carp counters for failures so it'll know > sending out the packet didn't work an become BACKUP. > We decided it would be better to log the error here instead of > failing in ip6_output. For me this seems to work! The patches did not apply cleanly to my 6.2-PRERELEASE, but I modified ip_carp.c by hand and left ip_fw2.c as it was or the kernel wouldn't build... with patches applied carp was fine! Great :-) Paul
Hi, I tested your latest patch and carp is working fine now ! on the mater : # ifconfig fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 options=8<VLAN_MTU> inet 130.79.201.191 netmask 0xffffff00 broadcast 130.79.201.255 inet6 2001:660:2402:1001::191 prefixlen 64 ether 00:02:b3:16:51:ed media: Ethernet autoselect (100baseTX <full-duplex>) status: active ... carp0: flags=49<UP,LOOPBACK,RUNNING> mtu 1500 inet 130.79.201.195 netmask 0xffffff00 inet6 2001:660:2402:1001::195 prefixlen 64 carp: MASTER vhid 198 advbase 1 advskew 0 on the slave : # ifconfig xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 options=9<RXCSUM,VLAN_MTU> inet 130.79.201.192 netmask 0xffffff00 broadcast 130.79.201.255 inet6 2001:660:2402:1001::192 prefixlen 64 ether 00:10:5a:66:dd:50 media: Ethernet autoselect (100baseTX <full-duplex>) status: active ... carp0: flags=49<UP,LOOPBACK,RUNNING> mtu 1500 inet 130.79.201.195 netmask 0xffffff00 inet6 2001:660:2402:1001::195 prefixlen 64 carp: BACKUP vhid 198 advbase 1 advskew 100 We can see the carp ipv6 advertisements on the slave from the master : # tcpdump -ni xl0 proto carp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on xl0, link-type EN10MB (Ethernet), capture size 96 bytes 16:57:47.849825 IP 130.79.201.191 > 224.0.0.18: VRRPv2, Advertisement, vrid 198, prio 0, authtype none, intvl 1s, length 36 16:57:47.849861 IP6 2001:660:2402:1001::191 > ff02::12: ip-proto-112 36 16:57:48.850729 IP 130.79.201.191 > 224.0.0.18: VRRPv2, Advertisement, vrid 198, prio 0, authtype none, intvl 1s, length 36 16:57:48.850758 IP6 2001:660:2402:1001::191 > ff02::12: ip-proto-112 36 16:57:49.851646 IP 130.79.201.191 > 224.0.0.18: VRRPv2, Advertisement, vrid 198, prio 0, authtype none, intvl 1s, length 36 16:57:49.851692 IP6 2001:660:2402:1001::191 > ff02::12: ip-proto-112 36 thanks a lot -- Philippe Pegon Bjoern A. Zeeb wrote: > After discussion with glebius I updated the patch to > a) plug an mbuf leak (yeah it was late last night;) > b) also increment the carp counters for failures so it'll know > sending out the packet didn't work an become BACKUP. > We decided it would be better to log the error here instead of > failing in ip6_output. > > Index: sys/netinet/ip_carp.c > =================================================================== > RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_carp.c,v > retrieving revision 1.27.2.8 > diff -u -p -r1.27.2.8 ip_carp.c > --- sys/netinet/ip_carp.c 25 Sep 2006 13:01:59 -0000 1.27.2.8 > +++ sys/netinet/ip_carp.c 6 Oct 2006 08:48:30 -0000 > @@ -965,9 +965,14 @@ carp_send_ad_locked(struct carp_softc *s > sizeof(struct in6_addr)); > /* set the multicast destination */ > > - ip6->ip6_dst.s6_addr8[0] = 0xff; > - ip6->ip6_dst.s6_addr8[1] = 0x02; > + ip6->ip6_dst.s6_addr16[0] = htons(0xff02); > ip6->ip6_dst.s6_addr8[15] = 0x12; > + if (in6_setscope(&ip6->ip6_dst, sc->sc_carpdev, NULL) != 0) { > + SC2IFP(sc)->if_oerrors++; > + m_freem(m); > + CARP_LOG("%s: in6_setscope failed\n", __func__); > + return; > + } > > ch_ptr = (struct carp_header *)(&ip6[1]); > bcopy(&ch, ch_ptr, sizeof(ch)); > Index: sys/netinet/ip_fw2.c > =================================================================== > RCS file: /shared/mirror/FreeBSD/r/ncvs/src/sys/netinet/ip_fw2.c,v > retrieving revision 1.106.2.20 > diff -u -p -r1.106.2.20 ip_fw2.c > --- sys/netinet/ip_fw2.c 20 Sep 2006 19:15:03 -0000 1.106.2.20 > +++ sys/netinet/ip_fw2.c 5 Oct 2006 21:47:33 -0000 > @@ -73,6 +73,7 @@ > #include <netinet/ip_fw.h> > #include <netinet/ip_divert.h> > #include <netinet/ip_dummynet.h> > +#include <netinet/ip_carp.h> > #include <netinet/pim.h> > #include <netinet/tcp.h> > #include <netinet/tcp_timer.h> > @@ -2315,6 +2316,16 @@ do { \ > PULLUP_TO(hlen, ulp, struct pim); > break; > > + case IPPROTO_CARP: > + PULLUP_TO(hlen, ulp, struct carp_header); > + if (((struct carp_header *)ulp)->carp_version != > + CARP_VERSION) + return > (IP_FW_DENY); > + if (((struct carp_header *)ulp)->carp_type != > + CARP_ADVERTISEMENT) + return > (IP_FW_DENY); > + break; > + > case IPPROTO_IPV6: /* RFC 2893 */ > PULLUP_TO(hlen, ulp, struct ip6_hdr); > break; >
Paul Dekkers wrote: > Bjoern A. Zeeb wrote: >> After discussion with glebius I updated the patch to >> a) plug an mbuf leak (yeah it was late last night;) >> b) also increment the carp counters for failures so it'll know >> sending out the packet didn't work an become BACKUP. >> We decided it would be better to log the error here instead of >> failing in ip6_output. > > For me this seems to work! > > The patches did not apply cleanly to my 6.2-PRERELEASE, but I modified > ip_carp.c by hand and left ip_fw2.c as it was or the kernel wouldn't > build... with patches applied carp was fine! Great :-) maybe a cut & paste effect, the patch at http://sources.zabbadoz.net/freebsd/patchset/20061005-01-carp-v6-scope-ipfw.diff apply cleanly > Paul -- Philippe Pegon
bz 2006-10-07 10:19:59 UTC FreeBSD src repository Modified files: sys/netinet ip_carp.c ip_fw2.c Log: Set scope on MC address so IPv6 carp advertisement will not get dropped in ip6_output. In case this fails handle the error directly and log it[1]. In addition permit CARP over v6 in ip_fw2. PR: kern/98622 Similar patch by: suz Discussed with: glebius [1] Tested by: Paul.Dekkers surfnet.nl, Philippe.Pegon crc.u-strasbg.fr MFC after: 3 days Revision Changes Path 1.44 +7 -2 src/sys/netinet/ip_carp.c 1.148 +11 -0 src/sys/netinet/ip_fw2.c _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
bz 2006-10-10 18:39:38 UTC FreeBSD src repository Modified files: (Branch: RELENG_6) sys/netinet ip_carp.c ip_fw2.c Log: MFC: 1.44 src/sys/netinet/ip_carp.c, 1.148 src/sys/netinet/ip_fw2.c > Set scope on MC address so IPv6 carp advertisement will not get dropped > in ip6_output. In case this fails handle the error directly and log it. > In addition permit CARP over v6 in ip_fw2. PR: kern/98622 Similar patch by: suz Discussed with: glebius [1] Tested by: Paul.Dekkers surfnet.nl, Philippe.Pegon crc.u-strasbg.fr Approved by: re (hrs) Revision Changes Path 1.27.2.9 +7 -2 src/sys/netinet/ip_carp.c 1.106.2.21 +11 -0 src/sys/netinet/ip_fw2.c _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
State Changed From-To: open->closed A changes has been comitted to HEAD and MFCed to RELENG_6 and will be part of 6.2R. Thanks for submitting and testing.
Guys, this panic with INVARIANTS/WITNESS in kernel and carp + ipv6 is still there, got it on yesterday's 9-STABLE. http://zhegan.in/files/carp-panic.jpg
State Changed From-To: closed->open Re-open; it's not the same backtrace but very similar problem.
Responsible Changed From-To: bz->gnn I shall not use bugzilla (at least until we will have a CLI).
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped
This is related to older implementation of CARP (before FreeBSD 10) and is not applicable to any supported FreeBSD version.