Tested on 11.0-RC1 with only vimage compiled into the kernel. Tested pf in vnet jail and no firewall on host. Tested pf in vnet jail and on the host. Vnet jail used this /etc/devfs.rules rule [devfsrules_vjail_pf=70] add include $devfsrules_jail add path pf unhide add path pfsync unhide add path pflog unhide Testing no pf firewall running on host, just in vnet jail. When starting vnet jail with pf, I check if pf kernel modules are loaded, if not them load them. Auto loading of modules does not happen. Issuing "ps ax" command on the host show this for pf, 925 - DL 0:00.35 [pf purge] Issuing "ifconfig -a" command from the started vnet jail console shows pflog0: flags=0<> metric 0 mtu 33184 groups: pflog The pf log file is allocated but not enabled in the vnet jail. Issuing the pf command "pfctl -sr -vv" from the started vnet jail console show this No ALTQ support in kernel ALTQ related functions disabled @0 block drop in quick on epair23b inet proto tcp from any to any port = nicname [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1301 State Creations: 0 ] @1 pass log (all) quick on epair23b all flags S/SA keep state [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1301 State Creations: 0 ] This at lease seems to indicate pf is running in the vnet jail. Issuing the "ping" command from the started vnet jail console works. Issuing the "whois" command from the started vnet jail console works also, but should not work because of the above block rule on port 43. This indicates that the pf rules in a vnet jail are not functioning. No pf log messages are posted in the vnet jail and no log messages are posted in the hosts pf log. Testing pf firewall running on host and vnet jail. After booting host, Issuing "ps ax" command on the host show this for pf, 393 - DL 0:00.07 [pf purge] 406 - Is 0:00.01 pflogd: [priv] (pflogd) 409 - S 0:00.00 pflogd: [running] -s 116 -i pflog0 -f /var/log/pflog (pflo Issuing "ifconfig -a" command from the started vnet jail console shows pflog0: flags=0<> metric 0 mtu 33184 groups: pflog The pf log file is allocated but not enabled in the vnet jail. Issuing the pf command "pfctl -sr -vv" from the started vnet jail console shows this No ALTQ support in kernel ALTQ related functions disabled @0 block drop in quick on epair23b inet proto tcp from any to any port = nicname [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1165 State Creations: 0 ] @1 pass log (all) quick on epair23b all flags S/SA keep state [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1165 State Creations: 0 ] Issuing the pf command "pfctl -sr -vv" from the host console shows this No ALTQ support in kernel ALTQ related functions disabled @0 pass log (all) quick on fxp0 all flags S/SA keep state [ Evaluations: 24 Packets: 9 Bytes: 740 States: 0 ] [ Inserted: uid 0 pid 411 State Creations: 3 ] The vnet jail results are the same as above. But the hosts pf log, logs this on vnet jail startup. pass out on fxp0: :: > ff02::16: HBH ICMP6, multicast listener report v2[|icmp6] pass out on epair23a: :: > ff02::16: HBH ICMP6, multicast listener report v2[|ic pass out on fxp0: :: > ff02::1:ff00:60b: ICMP6, neighbor solicitation[|icmp6] pass out on bridge0: :: > ff02::16: HBH ICMP6, multicast listener report v2[|icm pass out on fxp0: :: > ff02::16: HBH ICMP6, multicast listener report v2[|icmp6] pass out on epair23a: :: > ff02::16: HBH ICMP6, multicast listener report v2[|ic pass out on bridge0: :: > ff02::16: HBH ICMP6, multicast listener report v2[|icm pass out on fxp0: :: > ff02::16: HBH ICMP6, multicast listener report v2[|icmp6] pass out on fxp0: fe80::c1:ff:fe00:60b > ff02::16: HBH ICMP6, multicast listener Issuing the "ping" command from the started vnet jail console works and the hosts pf log shows this pass out on fxp0: 10.23.0.2 > 8.8.8.8: ICMP echo request pass in on fxp0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass in on bridge0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass out on fxp0: 10.23.0.2 > 8.8.8.8: ICMP echo request pass in on fxp0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass in on bridge0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass out on fxp0: 10.23.0.2 > 8.8.8.8: ICMP echo request pass in on fxp0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass in on bridge0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass out on fxp0: 10.23.0.2 > 8.8.8.8: ICMP echo request pass in on fxp0: 8.8.8.8 > 10.23.0.2: ICMP echo reply pass in on bridge0: 8.8.8.8 > 10.23.0.2: ICMP echo reply The hosts pf firewall is passing and logging all the traffic just as the host firewall single rule says to do. Issuing the "whois" command from the started vnet jail console works and it should not work because the vnet jail pf rules says to block all outbound traffic going to post 43. The hosts pf log shows this pass out on fxp0: 10.23.0.2.19180 > 199.212.0.46.43: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: pass out on fxp0: 10.23.0.2.19180 > 199.212.0.46.43: pass out on fxp0: 10.23.0.2.19180 > 199.212.0.46.43: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: pass out on fxp0: 10.23.0.2.19180 > 199.212.0.46.43: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on fxp0: 199.212.0.46.43 > 10.23.0.2.19180: pass in on bridge0: 199.212.0.46.43 > 10.23.0.2.19180: Issuing the pf command "pfctl -sr -vv" from the started vnet jail console shows that the counts have not increased. Another indicator that the pf firewall in the vnet jail is not functioning. Problems. 1. Why is the vnet jail issuing all that ipv6 traffic? It should only be generated if the vnet jail interface has a ipv6 ip address coded. 2. Why is ipv4 & ipv6 traffic making it to the host and NOT showing up on the epair23b interface? This is a problem if the host has a few vnet jails running at same time. IE: how am I going to control traffic on host to target correct vnet jail. In my case I use the epair number [23] in the ip address of the vnet jail. 3. Why are the pf rules in the vnet jail not being enforced? 4. Why does pf in the vnet jail not work when pf is not run on host? 5. Why does pf in the vnet jail not log to a log file in the vnet jails /var/log directory?
Just in reply to #3 as you say yourself in your description, it's outgoing packets, but your rule inside the jail specifies "in": 0 block drop in quick on epair23b inet proto tcp from any to any port = nicname Can you change that to "out" and see if it starts working? Currently on your "in" directions whois packets would originate from src port 43 and thus don't match the dest port 43.
I changed "in" to "out" in the vnet jail pf rules file. Here is the rules from inside of the vnet jail pfctl -sr -vv No ALTQ support in kernel ALTQ related functions disabled @0 block drop out quick on epair23b inet proto tcp from any to any port = nicnam e [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1171 State Creations: 0 ] @1 pass log (all) quick on epair23b all flags S/SA keep state [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1171 State Creations: 0 ] With pf on the host and in the vnet jail issuing the "whois" command from within the vnet jail still worked, and it should have not worked. The vnet pf firewall rules are not being enforced. Here is a snip it from the host pf log. pass out on fxp0: 10.23.0.2.29486 > 192.0.32.59.43: pass in on fxp0: 192.0.32.59.43 > 10.23.0.2.29486: pass in on bridge0: 192.0.32.59.43 > 10.23.0.2.29486: pass out on fxp0: 10.23.0.2.29486 > 192.0.32.59.43: pass in on fxp0: 192.0.32.59.43 > 10.23.0.2.29486: pass in on bridge0: 192.0.32.59.43 > 10.23.0.2.29486: pass out on fxp0: 10.23.0.2.29486 > 192.0.32.59.43: pass out on fxp0: 10.23.0.2.29486 > 192.0.32.59.43: pass in on fxp0: 192.0.32.59.43 > 10.23.0.2.29486: pass in on bridge0: 192.0.32.59.43 > 10.23.0.2.29486: pass in on fxp0: 192.0.32.59.43 > 10.23.0.2.29486: pass in on bridge0: 192.0.32.59.43 > 10.23.0.2.29486: In a net shell nothing changed from the first post. Those ipv6 packets are still being generated. The following is info for maybe debugging this problem. This is how I create the epair setup ifconfig ${nicname} alias 10.${vnetid}.0.1 ifconfig epair${vnetid} create ifconfig bridge0 addm epair${vnetid}a ifconfig epair${vnetid}a up This is the output of ifconfig -a command on the host after the vnet jail has started. /root >ifconfig -a fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 15 00 options=2009<RXCSUM,VLAN_MTU,WOL_MAGIC> ether 00:0c:f1:cd:55:ea inet 10.0.10.12 netmask 0xfffffff0 broadcast 10.0.10.15 inet 10.23.0.1 netmask 0xff000000 broadcast 10.255.255.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (100baseTX <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33184 groups: pflog bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:8f:94:84:0c:00 nd6 options=9<PERFORMNUD,IFDISABLED> groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: epair23a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 5 priority 128 path cost 2000 member: fxp0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 1 priority 128 path cost 200000 epair23a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mt u 1500 options=8<VLAN_MTU> ether 02:c1:00:00:05:0a inet6 fe80::c1:ff:fe00:50a%epair23a prefixlen 64 scopeid 0x5 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair Here is the output of ifconfig -a command issued from within the started vnet jail. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo pflog0: flags=0<> metric 0 mtu 33184 groups: pflog epair23b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 02:c1:00:00:06:0b inet 10.23.0.2 netmask 0xff000000 broadcast 10.255.255.255 inet6 fe80::c1:ff:fe00:60b%epair23b prefixlen 64 scopeid 0x3 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair
vnet is supported in 12, and pf works both on host and in vnet jails there.