Bug 148260 - [pf] [patch] pf rdr incompatible with dummynet
Summary: [pf] [patch] pf rdr incompatible with dummynet
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 7.2-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs (Nobody)
Depends on:
Reported: 2010-06-30 13:20 UTC by Alexey Guskov
Modified: 2017-12-31 22:32 UTC (History)
0 users

See Also:

file.diff (1.19 KB, patch)
2010-06-30 13:20 UTC, Alexey Guskov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Guskov 2010-06-30 13:20:07 UTC
The problem is the same as described here: http://lists.freebsd.org/pipermail/freebsd-net/2006-July/011076.html

I'm using two firewalls: ipfw for dummynet traffic shaping and pf for rules, forwards etc. When a dummynet pipe is used and rdr rule exists pf (i.e. for transparent proxy) tcp connections doesn't get established.

For example: we have a pipe for traffic going to user, and rdr rule for trasparent proxy. Ipfw processes incoming packets before pf. Here's how traffic goes thru kernel:

SYN packet:
[network interface] -> ... -> pfil_run_hooks() -> ipfw_check_in() -> pf_check_in() /the packet is being redirected and pf state entry is created/ -> ... -> [squid]

The packet is successfully delivered.

ACK packet:
[squid] -> ... -> ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /the packet is being unredirected using state table entry/ -> ipfw_check_out() -> ip_dn_io_ptr()

NB: ipfw stops processing the packet after it bin injected into dummynet. Heres what happens next:

ip_dn_ip_ptr() -> /some pipe vodoo magic/ -> dummynet_send() -> ip_output()

TADA! After coming out from pipe the packet is being re-injected into main packet-processing routine! Let's see what's gonna happen:

ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /pf cannot find entry in state table because the packet has already been unredirected and violently kills it/ -> OH SHI--

So, tcp connection never gets established.

Fix: When ipfw send a packet to dummynet it tags it with PACKET_TAG_DUMMYNET and saves rule number that triggered this action. When the packet comes to ipfw for the second time, ipfw can see that it already has been checked and continues processing from the next rule.

The problem is pf doesn't know that the packet already has been processed by him. So we'll teach him:
How-To-Repeat: pf.conf
rdr on le1 proto tcp from { } to any port { 80 } -> (le1) port 3128 

ipfw pipe 1 config bw 16KByte/s
ipfw 100 add pipe 1 all from any to

make ipfw process incoming packets before pf 
# ipfw disable firewall
# ipfw enable firewall 
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2010-07-03 12:17:25 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-pf

Over to maintainer(s).
Comment 2 mwisnicki 2011-03-25 21:52:22 UTC
How about a more generic solution:

Add new mbuf tag PACKET_TAG_PFIL_RESUME_FROM that contains address of
a function registered with pfil_add_hook (ipfw_check_hook in this
case) and modify pfil_run_hooks() to skip all hooks until that one (if
such tag is present).

Before reinjecting packet into ip_output by dummynet, prepend this
m_tag to mbuf (also strip that tag if it ever comes back?).

I don't know if mbuf api allows it but such tag could theoretically
have just one instance (created on dummynet module load) to avoid
allocation costs.

This way you don't have to put ugly workaround in every pfil consumer.
Comment 3 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:15 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped