IPv6 path mtu discovery is not working when pf is enabled and active. OS Version is 10.3 BETA1 A very similar bug was reported in 115413, and i see that the there mentioned patch is applied, but v6 pmtu is still not working. There was already a mail sent to freebsd-net some time ago with a detailed description of what is happening and things i tried, it can be found at https://lists.freebsd.org/pipermail/freebsd-net/2014-May/038590.html The problem is, that the machine in question sends out an (obviously) too large packet. The router (which is also a FreeBSD machine being the gateway to sixxs) detects this and sends an ICMP Packet too big back. In case pf is disabled, all is fine and you can see the updated mtu in net.inet.tcp.hostcache.list. As soon as you enable pf, this does not take place anymore and the router sends the ICMP too big messages 4 or 5 times and the connection hangs and no update to net.inet.tcp.hostcache.list is taking place. At the first time i saw this, it was a 2014 FreeBSD 10, now i upgraded the machine to 10.3 BETA1 and the effect is still the same. I made a patch to /usr/src/sys/netpfil/pf/pf.c with a shortcircuit and some printfs and then pmtu functions again with pf enabled (you see it in net.inet.tcp.hostcache.list) - i think this is not a fix but at least a workaround: *** pf.c-DIST Thu Jan 14 12:06:14 2016 --- pf.c Mon Feb 8 16:23:41 2016 *************** *** 4759,4766 **** pf_print_state(*state); printf(" seq=%u\n", seq); } ! REASON_SET(reason, PFRES_BADSTATE); ! return (PF_DROP); } else { if (V_pf_status.debug >= PF_DEBUG_MISC) { printf("pf: OK ICMP %d:%d ", --- 4759,4780 ---- pf_print_state(*state); printf(" seq=%u\n", seq); } ! if(icmptype != ICMP6_PACKET_TOO_BIG) ! { ! REASON_SET(reason, PFRES_BADSTATE); ! return (PF_DROP); ! } ! else ! { ! printf("pf: suspicious ICMP %d:%d ", ! icmptype, pd->hdr.icmp->icmp_code); ! pf_print_host(pd->src, 0, pd->af); ! printf(" -> "); ! pf_print_host(pd->dst, 0, pd->af); ! printf(" state: "); ! pf_print_state(*state); ! printf(" seq=%u\n", seq); ! } } else { if (V_pf_status.debug >= PF_DEBUG_MISC) { printf("pf: OK ICMP %d:%d ",
I can't seem to immediately reproduce this. Can you attach your pf.conf?
I'm still unable to reproduce this. The bug report (and the linked e-mail) suggest that the ICMP6 packet too big packet is rejected because the TCP header it contains has a sequence number which doesn't match the TCP connection. That's why the patch here seems to help: it bypasses the sequence number check. Clearly that's not what we want to do though. The first step is determining if the bug is in pf (incorrectly parsing the ICMP6 packet) or in the router/destination host (generating an incorrect IMCP6 packet). In order to debug this further I'll need the pf rules and a network capture demonstrating the problem.
(Assigned to reporter, because more information is required.)
After much searching for the cause of this bug, i found that the cause of this bug lies between my ears. The firewall between the two FreeBSD machines modifies the sequence numbers of the TCP packets flowing through it. So the sender machine has other sequence numbers than the receiving machine. But the TCP packet inside the ICMP "packet too big" frame contains the (incorrect) unmodified sequence numbers before the firewall. When pf is enabled, it checks the sequence number of the TCP frame inside the ICMP packet and discards it, if it is out of range. The FreeBSD kernel seems not to check the range and so PMTU discovery works without pf. The sending of the unmodified sequence number was caused by a "forgotten" (sic!) rule in the firewall, which allowed stateless passing of ICMP packets between the machines. After converting this to a stateful behaviour, TCP packets inside ICMP "packet too big" frames contain the (correct) modified sequence numbers and all is fine now with and without pf. Sorry for the false alarm, hellmuth