Bug 248088 - ipfilter variable substitution in rules & nat file not documented
Summary: ipfilter variable substitution in rules & nat file not documented
Status: In Progress
Alias: None
Product: Base System
Classification: Unclassified
Component: conf (show other bugs)
Version: 12.1-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: Cy Schubert
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-18 22:56 UTC by joeb1
Modified: 2020-07-30 02:53 UTC (History)
2 users (show)

See Also:


Attachments
IPF_PREDEFINED (633 bytes, patch)
2020-07-23 16:48 UTC, Cy Schubert
no flags Details | Diff
rc.d patch to export environment to ipf and ipnat (2.08 KB, patch)
2020-07-24 05:33 UTC, Cy Schubert
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description joeb1 2020-07-18 22:56:31 UTC
"man ipf.conf" manual section "variables" says it supports the use of variable substitution throughout the text of the file. Then it says "See ipf(8) for instructions on how to define variables to be used from a shell environment.

But man 8 ipf  contains NO such instructions.

Instructions need to be added to "man 8 ipf"

"man ipnat.conf" says it supports the use of variable substitution throughout the text of the file. Then it says "See ipnat(8) for instructions on how to define variables to be used from a shell environment.

But man 8 ipnat  contains NO such instructions.

Instructions need to be added to "man 8 ipnat"
Comment 1 Cy Schubert freebsd_committer 2020-07-23 15:15:48 UTC
You misunderstand. The variables are tunables. Many used to be sysctls. There is no variable expansion in rules like there is in pf.

The variables are there, i.e.

cwfw# ipf -T list | grep active
active	min 0	max 0	current 0
active	min 0	max 0	current 0
cwfw# ipf -T list | grep chksrc
chksrc	min 0	max 1	current 0
chksrc	min 0	max 1	current 0
cwfw# 

cwfw# ipf -T list
ipf_flags	min 0	max 4294967295	current 0
active	min 0	max 0	current 0
control_forwarding	min 0	max 1	current 0
update_ipid	min 0	max 1	current 0
chksrc	min 0	max 1	current 0
min_ttl	min 0	max 1	current 4
icmp_minfragmtu	min 0	max 1	current 68
default_pass	min 0	max 4294967295	current 134217730
tcp_idle_timeout	min 1	max 2147483647	current 864000
tcp_close_wait	min 1	max 2147483647	current 480
tcp_last_ack	min 1	max 2147483647	current 60
tcp_timeout	min 1	max 2147483647	current 480
tcp_syn_sent	min 1	max 2147483647	current 480
tcp_syn_received	min 1	max 2147483647	current 480
tcp_closed	min 1	max 2147483647	current 60
tcp_half_closed	min 1	max 2147483647	current 14400
tcp_time_wait	min 1	max 2147483647	current 480
udp_timeout	min 1	max 2147483647	current 240
udp_ack_timeout	min 1	max 2147483647	current 24
icmp_timeout	min 1	max 2147483647	current 120
icmp_ack_timeout	min 1	max 2147483647	current 12
ip_timeout	min 1	max 2147483647	current 120
ipf_flags	min 0	max 4294967295	current 0
active	min 0	max 0	current 0
control_forwarding	min 0	max 1	current 0
update_ipid	min 0	max 1	current 0
chksrc	min 0	max 1	current 0
min_ttl	min 0	max 1	current 4
icmp_minfragmtu	min 0	max 1	current 68
default_pass	min 0	max 4294967295	current 134217730
tcp_idle_timeout	min 1	max 2147483647	current 864000
tcp_close_wait	min 1	max 2147483647	current 480
tcp_last_ack	min 1	max 2147483647	current 60
tcp_timeout	min 1	max 2147483647	current 480
tcp_syn_sent	min 1	max 2147483647	current 480
tcp_syn_received	min 1	max 2147483647	current 480
tcp_closed	min 1	max 2147483647	current 60
tcp_half_closed	min 1	max 2147483647	current 14400
tcp_time_wait	min 1	max 2147483647	current 480
udp_timeout	min 1	max 2147483647	current 240
udp_ack_timeout	min 1	max 2147483647	current 24
icmp_timeout	min 1	max 2147483647	current 120
icmp_ack_timeout	min 1	max 2147483647	current 12
ip_timeout	min 1	max 2147483647	current 120
log_suppress	min 0	max 1	current 1
log_all	min 0	max 1	current 0
log_size	min 0	max 524288	current 32768
state_max	min 1	max 2147483647	current 4013
state_size	min 1	max 2147483647	current 5737
state_lock	min 0	max 1	current 0
state_maxbucket	min 1	max 2147483647	current 26
state_logging	min 0	max 1	current 1
state_wm_high	min 2	max 100	current 99
state_wm_low	min 1	max 99	current 90
state_wm_freq	min 2	max 999999	current 20
nat_lock	min 0	max 1	current 0
nat_table_size	min 1	max 2147483647	current 2047
nat_table_max	min 1	max 2147483647	current 30000
nat_rules_size	min 1	max 2147483647	current 127
rdr_rules_size	min 1	max 2147483647	current 127
hostmap_size	min 1	max 2147483647	current 2047
nat_maxbucket	min 1	max 2147483647	current 22
nat_logging	min 0	max 1	current 1
nat_doflush	min 0	max 1	current 0
nat_table_wm_low	min 1	max 99	current 90
nat_table_wm_high	min 2	max 100	current 99
frag_size	min 1	max 2147483647	current 257
frag_ttl	min 1	max 2147483647	current 120
proxy_debug	min 0	max 31	current 0
ftp_debug	min 0	max 127	current 0
ftp_pasvonly	min 0	max 1	current 0
ftp_insecure	min 0	max 1	current 0
ftp_pasvrdr	min 0	max 1	current 0
ftp_forcepasv	min 0	max 1	current 1
ftp_single_xfer	min 0	max 1	current 0
tftp_read_only	min 0	max 1	current 1
ftp_debug	min 0	max 127	current 0
ftp_pasvonly	min 0	max 1	current 0
ftp_insecure	min 0	max 1	current 0
ftp_pasvrdr	min 0	max 1	current 0
ftp_forcepasv	min 0	max 1	current 1
ftp_single_xfer	min 0	max 1	current 0
cwfw# 

Some of them are duplicated as sysctls.

cwfw# sysctl net.inet.ipf
net.inet.ipf.fr_ipfrttl: 120
net.inet.ipf.fr_defaultauthage: 600
net.inet.ipf.fr_authused: 0
net.inet.ipf.fr_authsize: 32
net.inet.ipf.ipf_hostmap_sz: 2047
net.inet.ipf.ipf_rdrrules_sz: 127
net.inet.ipf.ipf_natrules_sz: 127
net.inet.ipf.ipf_nattable_sz: 2047
net.inet.ipf.ipf_nattable_max: 30000
net.inet.ipf.fr_statemax: 4013
net.inet.ipf.fr_statesize: 5737
net.inet.ipf.fr_defnatage: 1200
net.inet.ipf.fr_minttl: 4
net.inet.ipf.fr_chksrc: 0
net.inet.ipf.fr_running: 1
net.inet.ipf.fr_icmptimeout: 120
net.inet.ipf.fr_udpacktimeout: 24
net.inet.ipf.fr_udptimeout: 240
net.inet.ipf.fr_tcpclosed: 60
net.inet.ipf.fr_tcptimeout: 480
net.inet.ipf.fr_tcplastack: 60
net.inet.ipf.fr_tcpclosewait: 480
net.inet.ipf.fr_tcphalfclosed: 14400
net.inet.ipf.fr_tcpidletimeout: 864000
net.inet.ipf.fr_active: 0
net.inet.ipf.ipf_pass: 134217730
net.inet.ipf.fr_flags: 0
cwfw# 

You have misunderstood the man pages.
Comment 2 Cy Schubert freebsd_committer 2020-07-23 15:58:11 UTC
It's poorly documented.

slippy# export IPF_PREDEFINED='abcd="keep state keep frags"'
slippy# ipf -ZFa -f /etc/ipf.conf
bad packets:		in 0	out 0
 input packets:		blocked 0 passed 2962 nomatch 0 counted 0
output packets:		blocked 48 passed 2913 nomatch 0 counted 0
 input packets logged:	blocked 0 passed 0
output packets logged:	blocked 48 passed 0
slippy# 
slippy# grep abcd /etc/ipf.conf
pass out quick on lagg0 proto icmp from any to any $abcd
pass out quick on bge0 proto icmp from any to any $abcd
pass out quick on wlan0 proto icmp from any to any $abcd
pass out quick on lagg0 proto tcp from any to any $abcd
pass out quick on bge0 proto tcp from any to any $abcd
pass out quick on wlan0 proto tcp from any to any $abcd
pass out quick on lagg0 proto udp from any to any $abcd
pass out quick on bge0 proto udp from any to any $abcd
pass out quick on wlan0 proto udp from any to any $abcd
slippy# 

I'll document it.

Personally, I use ipfmeta.
Comment 3 Cy Schubert freebsd_committer 2020-07-23 16:48:16 UTC
Created attachment 216714 [details]
IPF_PREDEFINED

Does this look ok?
Comment 4 commit-hook freebsd_committer 2020-07-23 17:40:03 UTC
A commit references this bug:

Author: cy
Date: Thu Jul 23 17:39:49 UTC 2020
New revision: 363453
URL: https://svnweb.freebsd.org/changeset/base/363453

Log:
  Document the IPFILTER_PREDEFINED environment variable.

  PR:		248088
  Reported by:	joeb1@a1poweruser.com
  MFC after:	1 week

Changes:
  head/contrib/ipfilter/man/ipf.8
Comment 5 joeb1 2020-07-24 00:37:13 UTC
You and I have different view of what variable substitution means. In the original PR I am talking about variable substitution in the rules file as in sh script to populate the scripting variables in the body of each rule, not sysctls variables.

The 2 ipf manuals seem to be talking about scripting variables. That is the documentation that is missing about how to accomplish that.
Comment 6 Cy Schubert freebsd_committer 2020-07-24 00:47:09 UTC
ipf doesn't support scripting but it does support simple substitution. Reply #2 shows how it can be done. You can pass ipfilter variables to ipf(8) using $IPF_PREDEFINED. I committed some documentation to the ipf man page to this effect.
Comment 7 joeb1 2020-07-24 00:52:14 UTC
Comment on attachment 216714 [details]
IPF_PREDEFINED

Need more documentation than this patch shows. Like for starters where is this IPF_PREDEFINED statement suppose to go? rc.conf??  Can there be more that one
IPF_PREDEFINED statement? Say one IPF_PREDEFINED="oif=re0"  followed be one for each shell type variable hard codded in the ipf.rules file.
Comment 8 Cy Schubert freebsd_committer 2020-07-24 01:10:46 UTC
Yes, except that you separate each variable assignment in IPF_PREDEFINED by semicolons.

You could put that in rc.conf for now, preceded by an export. However that would cause environment variable pollution, i.e. every process would would have that variable in its environment.

$ipf_predefined in rc.conf should export $IPF_PREDEFINED in rc.d/ipfilter & ipnat and unset after its use to avoid the pollution.

That should work for now, though I'm contemplating a more elegant solution which I'll discuss with Christos at NetBSD (we share ideas and code). I'll work on the rc.d scripts and post them here before committing them.
Comment 9 Cy Schubert freebsd_committer 2020-07-24 05:33:10 UTC
Created attachment 216737 [details]
rc.d patch to export environment to ipf and ipnat

Fill $ipf_predefined and $ipnat_predefined in rc.d to export IPF_PREDEFINED and IPNAT_PREDEFINED to ipf(8) and ipnat(8). The contents of the ipf_predefined and ipnat_predefined are the same as the environment variables.

I haven't tested this but it should work.
Comment 10 joeb1 2020-07-25 00:42:52 UTC
Since your open to discussing a method on how to integrate IPF_PREDEFINED & IPNAT_PREDEFINED into ipf(8) and ipnat(8) limiting the negative system impact that the current method of environment variables poses.

First consider the size of what info may be candidates for script substitution into the ipf.rules file. I could see the ipv6 prefix plus the unique portion which together can be on the large size. Then there are the other variables and all these variables together becomes a very large string.
'
I see this whole idea of export ipf_predefined="oif=re0,lif=em0" as barking up the wrong tree.

What I purpose is the solution used by pf. In the pf.rules file you place all your  
 variable definitions at the front of the rules file and the $variable name in the rules them selves. When the pf.rules file gets loaded the substitution occurs on the way to replacing the in core rules.
Comment 11 Cy Schubert freebsd_committer 2020-07-25 06:08:45 UTC
You've contradicted yourself. You can already put the variable definitions at the front of the file:

foo="lagg0";
bar="sk0";
pass in on $foo from any to any
pass out on $bar from any to any

Isn't this what you want? Because ipf already does this.

But if you want to script this in some way to dynamically pass $foo and $bar to ipf you must put them in $IPF_PREDEFINED, which I now understand you do not want. 

ipf already looks at IPF_PREDEFINED and IPNAT_PREDEFINED, but we don't use them in rc.d. (Which I understood at the time you wanted to dynamically generate them through a script.)

To summarize: If you want to use variables, just put them at the head of your ipf rules file.

(I suspect we may be talking cross purposes.)
Comment 12 commit-hook freebsd_committer 2020-07-30 02:53:24 UTC
A commit references this bug:

Author: cy
Date: Thu Jul 30 02:52:56 UTC 2020
New revision: 363685
URL: https://svnweb.freebsd.org/changeset/base/363685

Log:
  MFC r363453:

  Document the IPFILTER_PREDEFINED environment variable.

  PR:		248088
  Reported by:	joeb1@a1poweruser.com

Changes:
_U  stable/10/
  stable/10/contrib/ipfilter/man/ipf.8
_U  stable/11/
  stable/11/contrib/ipfilter/man/ipf.8
_U  stable/12/
  stable/12/contrib/ipfilter/man/ipf.8