At some point pf on FreeBSD switched the default behavior for this option so I had since removed it from the config of several of firewalls ... fragment reassemble Using scrub rules, fragments can be reassembled by normalization. In this case, fragments are buffered until they form a complete packet, and only the completed packet is passed on to the filter. The advantage is that filter rules have to deal only with complete packets, and can ignore fragments. The drawback of caching fragments is the additional memory cost. This is the default behaviour unless no fragment reassemble is specified. no fragment reassemble Do not reassemble fragments. However, while building a firewall using 14-RELEASE, I realized that fragmented IPsec ESP packets were not being re-assembled for processing by pf. After adding this line back into my pf.conf file and reloading, the traffic started flowing as expected ... scrub fragment reassemble My guess is that either the default behavior was reverted unintentionally or the the man page was never modified to match the new-new (old) behavior. Either way, it's very misleading.
I've been able to reproduce this issue. It happens in 14.0-RELEASE and 14.0-STABLE. It may be that pf is not processing (possibly dropping) fragmented packets by default. But I haven't checked beyond my ICMP example. This does appear fixed in 15.0-CURRENT, but you would need to check 15.0, mgrooms, and confirm. To reproduce, enable and start pf and pflog, use this pf.conf: block log proto icmp Then send a ping large enough to be fragmented to the pf host from an external host: ping -c 1 -s 2000 pfhost By default, in 13.2-RELEASE and 15.0-CURRENT, pf will log the blocked icmp request as two fragments. In 14.0-RELEASE and 14.0-STABLE, nothing gets logged. A regular small ping that doesn't get fragmented, ping -c 1 pfhost, will be logged. To log the fragmented packets in 14.0, you need to use "scrub fragment no reassemble" in pf.conf. You can log the reassembled packet in 14.0 with either the "set reassemble yes" option, or "scrub" or the equivalent "scrub fragment reassemble". Regarding documentation, the 14.0 pf.conf man page is missing the "set reassemble yes | no [no-df]" option description, however it is mentioned in the 14.0-RELEASE release notes: https://www.freebsd.org/releases/14.0R/relnotes/#network-general . The 15.0 man page has the new description. Also the pf.conf man page on both 14.0 and 15.0 for the pre-FreeBSD 14 scrub ruleset directive specifies using "no fragment reassemble", which does not work, "pfctl -n -f pf.conf" will throw a syntax error. The working syntax is "fragment no reassemble".
You might want to consider using the new OpenBSD-compatible syntax. Instead of using scrub rules, which are evaluated statelessly for each packet, you can enable fragment reassembly with a single "set reassemble yes" option at the top of pf.conf. There have been some updates to the man page to better describe the behaviour change, I don't think they got to FreeBSD 14.0, though. You are right, though, about behaviour change. The problem is that if scrub rules are not present, new syntax is in charge, and for this syntax the default is to not perform reassembly. The comment in the code is quite clear on the logic behind it: we expect people to still have the old style scrub rules in place. I've just missed the fact that scrub rules reassemble packets even when they are not present (Do they? I need to check that, I never relied on packet reassembly in my systems.) I'll talk with kp@ how to address it.
I appreciate the pf work on FreeBSD. I've deployed it extensively and use it daily. There is nothing wrong with the new default behavior from my perspective. It's just going to bite a lot of people who upgrade to 14 and rely on packet encapsulation due to crypto ( or other use cases ). That adds overhead which will causes fragmentation. The case was handled by default but now it must be explicitly enabled. And, since it was the default, I would guess that some users didn't know the option existed or what it was doing for them. Having outdated docs doesn't help. I wasn't aware of the newer pf syntax being implemented. Will have a look once the documentation is updated to match.
(In reply to mgrooms from comment #3) What version of FreeBSD were you using where the default behaviour worked with your IPSec flows? And before you added the "scrub fragment reassemble" config, did you have any "scrub" statements in the config, or no "scrub" config statements? (In reply to Kajetan Staszkiewicz from comment #2) For FreeBSD 14.0, I think using "scrub" rules may be a work-around to a broader issue. I think pf in 14.0 is not processing fragmented packets correctly. I tried another test, using an empty pf.conf (default pass rule). Monitoring the network interface with tcpdump, sending a large ping (2000 data bytes, so it will fragment). With pf running, the ping REQUEST is captured on the interface, but the host does not REPLY. If I repeat this with pf stopped, I get both REQUEST and REPLY. If I do the same test on 13.2 and 15.0, I get both REQUEST and REPLY with pf running. 14.0 is doing something different with the fragmented packets.
I looked some more into what happen to pf fragmentation between 13.2 and 14.0. The updates from this review: https://reviews.freebsd.org/D38025 made some changes to scrub and some other pf features. In file pf_norm.c, the default path for a fragmented packet changed from PASS to DROP. The condition on line 1090 in that file on 14.0-RELEASE (currently also the same line number on 14-STABLE) makes the decision to DROP. This has since been fixed on CURRENT (default behaviour will PASS a fragmented packet again). The updates were part of the following reviews: - fix to source code + tests: https://reviews.freebsd.org/D42355 - fix to documentation: https://reviews.freebsd.org/D42270 I think what is left is just getting the exiting updates from reviews D42355 and D42270 into 14-STABLE.
https://cgit.freebsd.org/src/commit/?h=stable/14&id=9f3032b76b3c077d032fb86b614a5d64e04a7ed4