Bugg in PF or in PF man-page? I vote for bug in PF itself… Man page says that “set skip on lo0” should ignore all traffic over lo0. This is not true. It just ignores 127* traffic. Let us assume the FreeBSD 11.2 host has IP 1.2.3.4. I created a jail and installed DNS/bind in it. The jail uses share IP with host (i.e no vnet recompiled kernel)… As there is no 127.0.0.1 I had to reconfigure rndc to listen and use 1.2.3.4 instead of 127.0.0.1 in the jail. I then noted that rndc did not work. In the FreeBSD main host pf.conf I had to explicit add a pass rule to allow 1.2.3.4 to 1.2.3.4 on lo0. Using tcpdump listening on lo0 I could also see the rndc traffic from 1.2.3.4 to 1.2.3.4 was going over lo0. So “set skip lo0” does not work as man page says which is… —snip— set skip on <ifspec> List interfaces for which packets should not be filtered. Packets passing in or out on such interfaces are passed as if pf was disabled, i.e. pf does not process them in any way. This can be useful on loopback and other virtual interfaces, when packet filtering is not desired and can have unexpected effects. For example: set skip on lo0 —snip— Now… I have not used FreeBSD that much. Especially not with jails. Have I missed something obvious and is too quick to log this? Otherwise, please enlighten me :)
Please include a full pf.conf, and ifconfig output. There are some issues around set skip handling in 11.2 (although I believe those are limited to set skip <group>). It's worth re-testing this on head.
vtnet0 has one IPv4 and and one IPv6 address on it. All jails use shared IP stack. No vnet... This is a snippet from the pf.conf in the main host. As this is a new host, I have not had the time to add variables into it... Let us assume the vtnet0 IPv4 IP is 1.2.3.4 --snip-- set skip on lo0 block all pass out quick on { lo0 vtnet0 } inet proto {tcp gre esp udp icmp ipv6} all keep state pass out quick on { lo0 vtnet0 } inet6 proto {tcp gre esp udp icmp6} all keep state pass out quick on { lo0 vtnet0 } inet6 all keep state -- REST OF PASS RULES CUT OUT -- pass in quick on lo0 inet proto tcp from 1.2.3.4 to 1.2.3.4 port 953 flags S/SAFR keep state --snip-- In the "DNS" jail, rndc refuse to work unless the last rule within the snippet above is added. So named.conf in the jail has controls to the 1.2.3.4 IP and rndc.conf has default listener conf set to 1.2.3.4 instead of 127.0.0.1 /Peo
'set skip' merely means do not run pf on this interface. It is not an 'allow all traffic from this interface' rule. In this case your 'set skip' rule means that traffic from lo0 is not blocked, but your block all rule is stopping it from being accepted on *vtnet0*, where your IP address is assigned. You do need the rule to actually accept traffic.
Hmmm From man page regarding "set skip".... --snip-- List interfaces for which packets should not be filtered. Packets passing in or out on such interfaces are passed as if pf was disabled, i.e. pf does not process them in any way. --snip-- I think the text is clear in the man page... Packets are passed as if PF was disabled. It also states that PF should not process them in any way. Two comments on this.. # 1 If a PF default block rule blocks traffic on lo0 for me when "set skip on lo0" on active. Then PF *IS* processing packages which the man page clearly say it should not! # 2 Lets assume you are right... Then the default block rule should also block 127.0.0.1 over lo0 so that as well have to be explicitly allowed. But it does not! So you comment.... Well... I do not agree. But please tell me if I misinterpret something. /Peo
Seems I cannot edit previous post. So here is an addition.. You say.. --snip-- but your block all rule is stopping it from being accepted on *vtnet0*, where your IP address is assigned. You do need the rule to actually accept traffic. --snip-- If you read my post I had to add a **pass in quick on lo0** an NOT vtnet0 Peo
Your packet goes out lo0 and in vtnet0. So without the pass rule it’s blocked.
I added this --ONLY-- to get it working... pass in quick on lo0 inet proto tcp from 1.2.3.4 to 1.2.3.4 port 953 flags S/SAFR keep state That is... Pass *IN* on lo0 and *NOT* vtnet0 So please explain your last comment so I understand. Regards Peo
I have now tested on my primary firewall that is OpenBSD 6.4. There I have "set skip on lo0". And I can in the firewall ping with the source IP of one of my interfaces and the target IP the set to the same. And I do not have to add a pass rule from that IP, to that IP, on lo0. On FreeBSD as stated I have to add a pass rule for incoming traffic on lo0 for this to work even though "set skip on lo0" is in the ruleset. So it works on OpenBSD... I guess that supports what I am saying. /Peo
(In reply to peos42 from comment #8) Please include full pf.conf, ifconfig output and command line. Have you tested this on head as well?
Have not tested on head. Is something fixed regarding this? Config posted below as requested. Note that IPv4 and IPv6 addresses are substituted to fake. ####################### ### FROM MAIN HOST #### ####################### 22:09:30 huey:~ # ifconfig -a vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:16:3c:7f:67:0e hwaddr 00:16:3c:7f:67:0e inet 1.2.3.4 netmask 0xffffff00 broadcast 1.2.3.255 inet6 fe80::216:3cff:fe7f:670e%vtnet0 prefixlen 64 scopeid 0x1 inet6 2222:3333:6:6df::1111 prefixlen 48 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160 groups: pflog 22:09:32 huey:~ # Note that the PF below will be rebuilt from scratch with variables and tagging etc. But for this case it doesn't matter.... 22:10:21 huey:~ # more /etc/pf.conf |grep -v ^#|sed '/^$/d' set skip on lo0 block all pass out quick on { lo0 vtnet0 } inet proto {tcp gre esp udp icmp ipv6} all keep state pass out quick on { lo0 vtnet0 } inet6 proto {tcp gre esp udp icmp6} all keep state pass out quick on { lo0 vtnet0 } inet6 all keep state antispoof quick for vtnet0 pass in log quick on vtnet0 inet proto icmp from any to vtnet0 icmp-type { 8 code 0 , 3 code 3 , 11 code 0 } keep state pass in quick on vtnet0 inet6 proto { ipv6-icmp } from any to any keep state block in log quick on vtnet0 proto tcp from <bad_hosts_SSH_MAIN_HOST> to vtnet0 port { 22 } pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 22 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_MAIN_HOST> flush global) pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 22 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_MAIN_HOST> flush global) block in log quick on vtnet0 proto tcp from <bad_hosts_SSH_DNS_HOST> to vtnet0 port { 10022 } pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 10022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_DNS_HOST> flush global) pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 10022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_DNS_HOST> flush global) pass in quick on vtnet0 inet proto tcp from any to vtnet0 port { 53 } flags S/SAFR keep state pass in quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 53 } flags S/SAFR keep state pass in quick on vtnet0 inet proto udp from any to vtnet0 port { 53 } keep state pass in quick on vtnet0 inet6 proto udp from any to vtnet0 port { 53 } keep state pass in quick on lo0 inet proto tcp from 1.2.3.4 to 1.2.3.4 port 953 flags S/SAFR keep state block in log quick on vtnet0 proto tcp from <bad_hosts_SSH_MAIL_HOST> to vtnet0 port { 20022 } pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 20022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_MAIL_HOST> flush global) pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 20022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_MAIL_HOST> flush global) pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 25 465 587 } flags S/SAFR keep state pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 25 465 587 } flags S/SAFR keep state block in log quick on vtnet0 proto tcp from <bad_hosts_SSH_WEB_HOST> to vtnet0 port { 30022 } pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 30022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_WEB_HOST> flush global) pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 30022 } flags S/SAFR keep state (max 100, max-src-nodes 30, max-src-states 30, max-src-conn 20, max-src-conn-rate 3/30, overload <bad_hosts_SSH_WEB_HOST> flush global) pass in log quick on vtnet0 inet proto tcp from any to vtnet0 port { 80 443 } flags S/SAFR keep state pass in log quick on vtnet0 inet6 proto tcp from any to vtnet0 port { 80 443 } flags S/SAFR keep state 22:10:24 huey:~ # ########################### ### FROM DNS JAIL HOST #### ############################ "rndc reload" does NOT work in this jail if the following pf.conf row is removed from the main host... pass in quick on lo0 inet proto tcp from 1.2.3.4 to 1.2.3.4 port 953 flags S/SAFR keep state On OpenBSD this is not needed as "set skip on lo0" works... But all this I have already written in earlier posts. 22:11:25 DNS:~ # ifconfig -a vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:16:3c:7f:67:0e hwaddr 00:16:3c:7f:67:0e inet 1.2.3.4 netmask 0xffffff00 broadcast 1.2.3.255 inet6 2222:3333:6:6df::1111 prefixlen 48 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160 groups: pflog 22:11:27 DNS:~ # 22:13:24 DNS:~ # more /usr/local/etc/namedb/rndc.conf |grep default-server default-server 1.2.3.4; 22:13:25 DNS:~ # 22:13:26 DNS:~ # more /usr/local/etc/namedb/named.conf |grep 953 inet 1.2.3.4 port 953 allow { 1.2.3.4; 127.0.0.1; 2222:3333:5:6df::1111; } keys { "rndc-key"; }; 22:13:31 DNS:~ #
(In reply to peos42 from comment #10) Yes, there have been changes around set skip handling (mostly for groups). See comment #1. Do you have a smaller test case?
Hi Kristof This is small :) However... To try to give you something smaller I started by removing the pass in quick on lo0 inet proto tcp from 1.2.3.4 to 1.2.3.4 port 953 flags S/SAFR keep state from the main host pf.conf and reloaded PF. The weird thing is that "rndc relosd" still works in the jail. I restarted the jail and it still works. As it should!!! The ONLY thing I have done since my initial post where it didn't work is to upgrade host and jail from 11.2-p4 to 11.2p5. ?????????????