For background, see my post to the freebsd-pf mailing list: http://docs.freebsd.org/cgi/mid.cgi?38ce25da0901271659m2b6d8a1fg2e425df93781f6f5 When an IP configured on a carp(4) interface is used on a gif(4) tunnel, in order that the endpoint of the tunnel fails over, the advskew setting for this carp interface is ignored and the interface is brought up with advskew 0. This obviously prevents proper operation of a "preferred master" configuration. After configuring devd to ignore attach events for carp interfaces (devd automatically runs "/etc/rc.d/netif start carp0" after you do an "ifconfig carp0 create", and does so needlessly after carp0 is already configured at boot time; that's a separate bug I shall file shortly), I have determined that the "start all interfaces" mode of /etc/rc.d/netif (i.e. "rc.d/netif start", without specifying an interface) configures the advskew correctly. It's the "rc.d/netif start carp0" mode that loses the advskew setting. In the standard configuration, "rc.d/netif start" is called and carp interfaces are brought up (correctly). A bit later in the bootup, devd starts, and thinks it needs to configure carpN, so it calls "rc.d/netif start carp0", which resets advskew to 0. I'm thinking it's a kernel bug since enabling rc_debug and logging the outputs of "sh -x /etc/rc.d/netif start" vs. "ifconfig carp0 create; sh -x /etc/rc.d/netif start carp0" shows that the exact same sequence of "ifconfig carp0 ..." commands is run. The only difference is that in the global start mode, other interfaces (including gif0) are ifconfig'd between "ifconfig carp0 create" and the rest of the "ifconfig carp0 ..." commands. I have observed this problem on both 7.1/amd64 and 6.2/i386. Fix: A workaround is to configure devd to ignore attach events for carp interfaces, like so: ==== # don't have devd call network_start on carp0; for some reason this # loses the advskew setting (besides being entirely superfluous). notify 100 { match "system" "IFNET"; match "subsystem" "carp[0-9]+"; match "type" "ATTACH"; action "echo devd called on $subsystem, type $type"; }; ==== This overrides the following section of devd.conf: notify 0 { match "system" "IFNET"; match "type" "ATTACH"; action "/etc/pccard_ether $subsystem start"; }; After doing this, carp interfaces are brought up correctly by "rc.d/netif start" on boot, and devd does not call "rc.d/netif start carp0", so the advskew setting remains correct. This doesn't fix the underlying problem but it is an acceptable workaround to me. How-To-Repeat: Assign the same IP to a carp interface and gif interface, e.g.: cloned_interfaces="carp0 carp1" gif_interfaces="gif0" carp0_advskew="advskew 100" ifconfig_carp0="vhid 16 $carp_advskew pass XXXXXX 10.95.8.1/21" ifconfig_gif0="10.95.8.1 10.95.0.1 netmask 255.255.255.255" The name "carp0" is nothing special; I can call it carp3 and the behavior is the same. Only the 'ifconfig_gif0' setting matters; the gifconfig_gif0 setting doesn't (an IP I used for that is configured on carp1, which has no problems). Also, the mere presence of gif0 doesn't matter -- when gif0 is configured with IPs that are NOT assigned to a carp interface, carp0 comes up with the correct advskew setting.
Responsible Changed From-To: freebsd-bugs->freebsd-net Over to maintainer(s).
Having the same issue here and the workaround isn't working for me -- it preserves the advskew but the vpn doesn't seem to work without the devd attach event. Hope someone will fix this soon.
Andrew, can you please share the entire configuration on all involved boxes: local and remote side of tunnel, the CARP master and slave boxes? I'd like to reproduce your configuration and check whether my new CARP implementation is subject to your problem or not. For more info, see: http://people.freebsd.org/~glebius/newcarp/ -- Totus tuus, Glebius.
Responsible Changed From-To: freebsd-net->glebius I'm now working on major rewrite of CARP for FreeBSD 10, and I'd like to take all related PRs.
State Changed From-To: open->closed Feedback timeout.
Sorry, I missed your earlier email. I am sending you my config off-list.
Add to Audit-Trail. ----- Forwarded message from "Daugherity, Andrew W" <adaugherity@tamu.edu> ----- Date: Fri, 16 Dec 2011 17:37:34 +0000 From: "Daugherity, Andrew W" <adaugherity@tamu.edu> To: "glebius@FreeBSD.org" <glebius@FreeBSD.org> Subject: carp config (kern/132107) Sorry for not responding earlier. Here's my config for a carp failover pair, which is one end of the gif tunnel. The other end of the tunnel is another pair configured very similarly. I abstracted as much as possible into rc.common.conf which I keep synced between the two. ipsec etc. settings have no effect on the this bug and can be ignored. I am now running 8.2 all around (this pair is amd64, the other i386); I haven't checked whether it behave any differently than 6.x/7.x did without my workaround (I would presume so, unless there are some carp changes in the kernel I don't know about) but the workaround still works and doesn't cause any problems. Kernel config: ==== % uname -a FreeBSD imslb3.tamu.edu 8.2-RELEASE-p4 FreeBSD 8.2-RELEASE-p4 #9: Mon Oct 31 15:27:42 CDT 2011 root@imslb3.tamu.edu:/usr/obj/usr/src/sys/IMS amd64 % cat /usr/src/sys/amd64/conf/IMS # include the GENERIC config (as of 7.0, GENERIC has SMP support, and there # are no more separate GENERIC and SMP kernels) include GENERIC ident VPR-SMP-GENERIC device carp device pf device pflog device pfsync # In order to enable IPSEC you MUST also add device crypto to # your kernel configuration options IPSEC #IP security (requires device crypto) #options IPSEC_DEBUG #debug for IP security device crypto ==== rc.confs attached. It's nearly completely intact for your benefit, but please scrub public IPs and hostnames (including in comments) before sharing publicly. Thanks, Andrew Daugherity Systems Analyst Division of Research, Texas A&M University adaugherity@tamu.edu ----- End forwarded message ----- -- Totus tuus, Glebius.
State Changed From-To: closed->open Feedback received.
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped