9.1-RELEASE kernel with modules and vimage plus ipfw compiled in. vnet jails running ipfw are logging to the host security file and don't log any ipfw log messages to the hosts message file. Secondly the vnet jails security and messages files never get populated with ipfw log messages. logger command works. logged msg in both security and messages on host vnet jail can ping the public internet. Hosts security file has log messages from both jail and host. ipfw log messages are not being put into the hosts messages file. Kernel compile options used. options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT # ran on the host # /root >sysctl net.inet.ip.fw.verbose net.inet.ip.fw.verbose: 1 # /root >sysctl net.inet.ip.fw.verbose_limit net.inet.ip.fw.verbose_limit: 0 # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any # /root >/var/log/security empty file # /root >cat /var/log/messages empty file # /root >logger -p security.notice host logger cmd 1 # /root >cat /var/log/security May 2 19:45:51 fbsdjones root: host logger cmd 1 # /root >cat /var/log/messages May 2 19:45:51 fbsdjones root: host logger cmd 1 # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any # /root >ping -c 3 freebsd.org PING freebsd.org (8.8.178.135): 56 data bytes 64 bytes from 8.8.178.135: icmp_seq=0 ttl=51 time=85.032 ms 64 bytes from 8.8.178.135: icmp_seq=1 ttl=51 time=84.381 ms 64 bytes from 8.8.178.135: icmp_seq=2 ttl=51 time=84.647 ms --- freebsd.org ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 84.381/84.687/85.032/0.267 ms # /root >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 9 869 allow log ip from any to any via rl0 65535 1 328 allow ip from any to any vnet jail started usinf jail(8) # /root >jls JID IP Address Hostname Path 1 - vdir2 /usr/jails/vdir2 # /root >jexec vdir2 tcsh vdir2 / >cat /etc/ipfw.rules # Flush out the list before we begin. ipfw -q -f flush cmd="ipfw -q add" if [ -e /etc/epair ]; then pif=`cat "/etc/epair"` rm /etc/epair else pif="lo0" fi $cmd 010 allow all from any to any via lo0 $cmd 011 allow log all from any to any via $pif vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 0 0 allow log ip from any to any via epair1b 65535 8 624 allow ip from any to any vdir2 / >ping -c 3 freebsd.org PING freebsd.org (8.8.178.135): 56 data bytes 64 bytes from 8.8.178.135: icmp_seq=0 ttl=51 time=84.342 ms 64 bytes from 8.8.178.135: icmp_seq=1 ttl=51 time=84.195 ms 64 bytes from 8.8.178.135: icmp_seq=2 ttl=51 time=84.015 ms --- freebsd.org ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 84.015/84.184/84.342/0.134 ms vdir2 / >ipfw -a list 00010 0 0 allow ip from any to any via lo0 00011 8 634 allow log ip from any to any via epair1b 65535 8 624 allow ip from any to any vdir2 / >cat /var/log/security May 1 21:56:27 vdir2 newsyslog[5202]: logfile first created vdir2 / >cat /var/log/messages May 1 21:56:27 vdir2 newsyslog[5202]: logfile first created vdir2 / >exit exit Back on the host # /root >cat /var/log/security May 2 19:45:51 fbsdjones root: host logger cmd 1 May 2 19:46:53 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.1:138 10.0.10.7:138 in via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept UDP 10.0.10.5:64721 209.18.47.61:53 out via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.0.10.5:64721 in via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:46:58 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:46:59 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:46:59 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:47:00 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.0.10.5 8.8.178.135 out via rl0 May 2 19:47:00 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.0.10.5 in via rl0 May 2 19:47:38 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:47:38 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:47:39 fbsdjones kernel: ipfw: 11 Accept ICMPv6:135.0 [::] [ff02::1:ff00:b0b] out via rl0 May 2 19:47:39 fbsdjones kernel: ipfw: 11 Accept ICMPv6:143.0 [::] [ff02::16] out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 10.1.0.2:13101 209.18.47.61:53 out via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 10.1.0.2:13101 209.18.47.61:53 out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept UDP 209.18.47.61:53 10.1.0.2:13101 in via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:21 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:22 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via epair1b May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:8.0 10.1.0.2 8.8.178.135 out via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via rl0 May 2 19:49:23 fbsdjones kernel: ipfw: 11 Accept ICMP:0.0 8.8.178.135 10.1.0.2 in via epair1b # /root >cat /var/log/messages May 2 19:45:51 fbsdjones root: host logger cmd 1 May 2 19:47:38 fbsdjones kernel: bridge0: Ethernet address: 02:8f:94:84:0c:00 May 2 19:47:38 fbsdjones kernel: bridge0: link state changed to UP May 2 19:47:38 fbsdjones kernel: epair1a: Ethernet address: 02:c0:24:00:0a:0a May 2 19:47:38 fbsdjones kernel: epair1b: Ethernet address: 02:c0:24:00:0b:0b May 2 19:47:38 fbsdjones kernel: epair1a: link state changed to UP May 2 19:47:38 fbsdjones kernel: epair1b: link state changed to UP May 2 19:50:59 fbsdjones kernel: epair1a: link state changed to DOWN May 2 19:50:59 fbsdjones kernel: epair1b: link state changed to DOWN May 2 19:50:59 fbsdjones kernel: bridge0: link state changed to DOWN May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (30 items). Lost 2 pages of memory. May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (203 items). Lost 1 pages of memory. May 2 19:51:02 fbsdjones kernel: Freed UMA keg was not empty (30 items). Lost 2 pages of memory. May 2 19:51:02 fbsdjones kernel: hhook_vnet_uninit: hhook_head type=1, id=1 cleanup required May 2 19:51:02 fbsdjones kernel: hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required # /root >exit exit
Responsible Changed From-To: freebsd-bugs->freebsd-ipfw Over to maintainer(s).
> 9.1-RELEASE kernel with modules and vimage plus ipfw compiled in. > vnet jails running ipfw are logging to the host security file and > don't log any ipfw log messages to the hosts message file. Secondly > the vnet jails security and messages files never get populated with > ipfw log messages. Logging to the host's syslog rather than the jail's appears to be the main/real issue here, confirmed and demonstrated by Anders Hagman, see http://lists.freebsd.org/pipermail/freebsd-ipfw/2013-May/005398.html > logger command works. logged msg in both security and messages on > host > vnet jail can ping the public internet. > Hosts security file has log messages from both jail and host. > ipfw log messages are not being put into the hosts messages file. Apart from certain admin messages such as ipfw initialisation, 'limit N reached on rule X' and 'Entry X logging count reset.' ipfw log messages are never written to /var/log/messages but only to /var/log/security. Since you set verbose_limit=0, you shouldn't expect to see anything from ipfw in /var/log/messages, on either host or jail. > # /root >/var/log/security > empty file > > # /root >cat /var/log/messages > empty file Strange that there were not even normal bootup messages on the host? The rest serves to demonstrate the vnet jail logging-to-host issue. Ian
Ian Smith wrote: > > > 9.1-RELEASE kernel with modules and vimage plus ipfw compiled in. > > vnet jails running ipfw are logging to the host security file and > > don't log any ipfw log messages to the hosts message file. Secondly > > the vnet jails security and messages files never get populated with > > ipfw log messages. > > Logging to the host's syslog rather than the jail's appears to be the > main/real issue here, confirmed and demonstrated by Anders Hagman, see > http://lists.freebsd.org/pipermail/freebsd-ipfw/2013-May/005398.html You have the incorrect conclusion. Let me reword what was stated in the original pr to give a clearer picture of the pr. IPFW log messages coming from a IPFW process running inside of a jail(8) vnet jail are being written to the hosts /etc/log/security file and not to the vnet jail's /etc/log/security file. If the host is also running ipfw, it's logging messages are intermingled with those coming from the vnet jail ipfw process. And yes Anders Hagman did confirm this per the link you provided. > > > logger command works. logged msg in both security and messages on > > host > > vnet jail can ping the public internet. > > Hosts security file has log messages from both jail and host. > > ipfw log messages are not being put into the hosts messages file. > > Apart from certain admin messages such as ipfw initialization, 'limit N > reached on rule X' and 'Entry X logging count reset.' ipfw log messages > are never written to /var/log/messages but only to /var/log/security. > Since you set verbose_limit=0, you shouldn't expect to see anything from > ipfw in /var/log/messages, on either host or jail. I don't know how you can to that conclusion. verbose_limit is not mentioned in this pr. You are incorrect. verbose_limit is not set for this pr test. > > > # /root >/var/log/security > > empty file > > > > # /root >cat /var/log/messages > > empty file > > Strange that there were not even normal bootup messages on the host? Thats because I deleted all content before running this test to make the output simple. What purpose would showing boot messages serve? > > The rest serves to demonstrate the vnet jail logging-to-host issue. > > Ian > >
> You have the incorrect conclusion. Let me reword what was stated in the > original pr to give a clearer picture of the pr. IPFW log messages coming > from a IPFW process running inside of a jail(8) vnet jail are being written > to the hosts /etc/log/security file and not to the vnet jail's > /etc/log/security file. Exactly so; if rewording what I said assists comprehension, fine. > If the host is also running ipfw, it's logging > messages are intermingled with those coming from the vnet jail ipfw process. > And yes Anders Hagman did confirm this per the link you provided. Again, exactly so. Anders used different rule numbers on host and jail which made following the log easier, but your example log is followable. > > Since you set verbose_limit=0, you shouldn't expect to see anything from > > ipfw in /var/log/messages, on either host or jail. > > I don't know how you can to that conclusion. verbose_limit is not mentioned > in this pr. You are incorrect. verbose_limit is not set for this pr test. Sigh. Paragraphs 4 and 5 of _this_ PR: # ran on the host # /root >sysctl net.inet.ip.fw.verbose net.inet.ip.fw.verbose: 1 # /root >sysctl net.inet.ip.fw.verbose_limit net.inet.ip.fw.verbose_limit: 0 > > Strange that there were not even normal bootup messages on the host? > > Thats because I deleted all content before running this test to make the > output simple. What purpose would showing boot messages serve? You may find tail(1) useful. Ian
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped
Now testing 12.0-RC3. vnet jail running on the gateway host. IPF firewall running on the gateway host and ipfw running in the vnet jail. Found out about the undocumented ipfw0 log. This works in each vnet jail logging the vnet jails log records to the /var/log/security.log file in the vnet jail. To enable place the normal ipfw statements in the vnet jails rc.conf with these changes. firewall_logging="NO" firewall_logif="YES" nohup tcpdump -lnti ipfw0 | logger -t jailname -p security.info & This method should be documented someplace.
RELEASE 12.1 host and vnet jail both running ipfw. Vnet jail ipfw log records being written into the hosts /var/log/security file interspersed with the host ipfw log records. Same problem since reported release 9.1. That's 3 manager release version this bug has been around. The vnet jails ipfw log records should be written to the /var/log/security file in the vnet jail, not exiting the the vnet jail container to be written on the host.