With a pair of pfsync and carp firewalls some times, perhaps about 5% to 10%
	of the time, some TCP sessions will not transfer during a transition of
	the carp master. The error, if pf loud debugging is enabled, is a number
	of kernel messages like:
	    pf: State failure on: 1       |
	    pf: BAD state: TCP 67.134.74.10:53846 67.134.74.10:53846 207.109.234.245:80 [lo=3086765454 high=3086830989 win=65535 modulator=0] [lo=310734435 high=310734435 win=65535 modulator=0] 4:4 A seq=310747575 ack=3086765454 len=1460 ackskew=0 pkts=82511:161080 dir=in,rev
	Some more detail can be found at 
	    http://lists.freebsd.org/pipermail/freebsd-pf/2006-July/002317.html

	The symptom is that TCP sessions will simply hang if they are not
	transferred correctly.

	The chance of a TCP stream not transferring seems to be directly
	related to the speed of the stream. We have not experienced the problem
	with essentially idle, or interactive streams. But transfers of large
	files from fast sites fairly often experience the problem.

Fix: 

While not a fix, this RC script in /usr/local/etc/rc.d seems to work
	around the problem most of the time. But we still experience stuck
	TCP sessions even with this script.


#!/bin/sh
#

# PROVIDE: carp
# REQUIRE: NETWORKING
# KEYWORD: nojail shutdown

. /etc/rc.subr
. /etc/network.subr

name="carp"
start_cmd="carp_start"
stop_cmd="carp_stop"
_interfaces="`ifconfig -l | tr ' ' '\n' | grep '^carp[0-9]'`"
startup_delay=90
advskew_up=0
advskew_down=100

carp_start()
{
	echo "Changing advskew on carp interfaces to ${advskew_up} in ${startup_delay} seconds."
	(
		sleep ${startup_delay}
		echo -n "Changing advskew on carp interfaces to ${advskew_up}:" > /dev/console
		for ifn in ${_interfaces}; do
			ifconfig ${ifn} advskew $advskew_up
			echo -n " ${ifn}" > /dev/console
		done
		echo "." > /dev/console
	) &
}

carp_stop()
{
	echo -n "Changing advskew on carp interfaces to ${advskew_down}:"
	for ifn in ${_interfaces}; do
		ifconfig ${ifn} advskew $advskew_down
		echo -n " ${ifn}"
	done
	echo '.'
}

load_rc_config $name
run_rc_command $*
How-To-Repeat: 	Setup a pair of pfsync/carp firewalls, do some high-speed transfers
	across the firewalls, and repeatedly reboot the master.