Bug 148260

The problem is the same as described here: http://lists.freebsd.org/pipermail/freebsd-net/2006-July/011076.html

I'm using two firewalls: ipfw for dummynet traffic shaping and pf for rules, forwards etc. When a dummynet pipe is used and rdr rule exists pf (i.e. for transparent proxy) tcp connections doesn't get established.

For example: we have a pipe for traffic going to user, and rdr rule for trasparent proxy. Ipfw processes incoming packets before pf. Here's how traffic goes thru kernel:

SYN packet:
[network interface] -> ... -> pfil_run_hooks() -> ipfw_check_in() -> pf_check_in() /the packet is being redirected and pf state entry is created/ -> ... -> [squid]

The packet is successfully delivered.

ACK packet:
[squid] -> ... -> ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /the packet is being unredirected using state table entry/ -> ipfw_check_out() -> ip_dn_io_ptr()

NB: ipfw stops processing the packet after it bin injected into dummynet. Heres what happens next:

ip_dn_ip_ptr() -> /some pipe vodoo magic/ -> dummynet_send() -> ip_output()

TADA! After coming out from pipe the packet is being re-injected into main packet-processing routine! Let's see what's gonna happen:

ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /pf cannot find entry in state table because the packet has already been unredirected and violently kills it/ -> OH SHI--

So, tcp connection never gets established.

Fix: When ipfw send a packet to dummynet it tags it with PACKET_TAG_DUMMYNET and saves rule number that triggered this action. When the packet comes to ipfw for the second time, ipfw can see that it already has been checked and continues processing from the next rule.

The problem is pf doesn't know that the packet already has been processed by him. So we'll teach him:
How-To-Repeat: pf.conf
--
rdr on le1 proto tcp from { 192.168.1.0/24 } to any port { 80 } -> (le1) port 3128 
--

ipfw.conf
--
ipfw pipe 1 config bw 16KByte/s
ipfw 100 add pipe 1 all from any to 192.168.1.101
--

make ipfw process incoming packets before pf 
--- 
# ipfw disable firewall
# ipfw enable firewall 
--