Bug 263974
| Summary: | ipfw_nat64lsn reply destination mac address error | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Jim B. <jpb> | ||||||
| Component: | kern | Assignee: | freebsd-ipfw (Nobody) <ipfw> | ||||||
| Status: | Closed FIXED | ||||||||
| Severity: | Affects Some People | CC: | ae, zarychtam | ||||||
| Priority: | --- | ||||||||
| Version: | 13.0-RELEASE | ||||||||
| Hardware: | amd64 | ||||||||
| OS: | Any | ||||||||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255928 | ||||||||
| Attachments: |
|
||||||||
Setting net.inet.ip.fw.nat64_direct_output=0 helped in my case. I replaced the FreeBSD 13.0-p11 firewall with an instance of the BSDP Router Project running on Qemu. BSDRP Platform: FreeBSD router.bsdrp.net 14.0-CURRENT FreeBSD 14.0-CURRENT BSDRP-AMD64 amd64 The architecture and ruleset were unchanged. The nat64lsn instance works correctly and the communication is successful. sysctl net.inet.ip.fw.nat64_direct_output=0 is required for the ruleset shown. It seems that the problem is narrowed to ipfw_nat64 13.0 code. I will check again with FreeBSD 13.1 when it drops. Jim B. This was probably fixed with https://reviews.freebsd.org/D31680 Created attachment 233966 [details] nat64lsn on my addressing scheme not working In my comment #2 I explained that I substituted the BSD Router Project instance and got the example working with the addressing scheme layout shown here: https://bsdrp.net/documentation/examples/nat64 But when I reconfigured my example to the layout shown in this attachment (nat64lsn_myaddr_notworking.png) the nat64lsn instance *stopped* working. I'm going to spend some time going over both configurations with a fine tooth comb to ensure I didn't miss anything, but for now, I'll say I still have a problem. Ok, Comment #4 is solved by use of the "allow_private" keyword on the nat64lsn create statement. While the ipfw(8) man page mentions that IPv4 RFC 1918 addresses are not processed, it does not mention RFC 5737 addresses (192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24 - the one I was using. Adding the "allow_private" keyword fixed that for the use of the BSD Router Project instance described in Comment #4. However, going back and using my original 13.0-RELEASE-p11 VM instance, the addition of the "allow_private" keyword does not fix the duplicated source MAC address in the destination field in the reply. Now that 13.1 has dropped, I will spin up a VM and test with that. Ok, as promised in Comment #5, I spun up a new VM with 13.1-RELEASE and retested this issue. I now report that nat64lsn (stateful NAT64) *does work* under the following conditions: * The addressing scheme is as described in the attachment "nat64lsn on my addressing scheme not working" now *does work*. * The ipfw rules to implement are: ipfw nat64lsn foo create prefix4 203.0.112.0/24 allow_private ipfw add allow log ipv6-icmp from any to any icmp6types 135,136 ipfw add nat64lsn foo log ip from 2001:db8:12::/64 to 64:ff9b::/96 in ipfw add nat64lsn foo log ip from any to 203.0.112.0/24 in ipfw add allow log ip from any to any * The direct_output sysctl had to be set to 1 (not zero): sysctl net.inet.ip.fw.nat64_direct_output=1 * I also set the nat64_debug sysctl and the firewall verbose sysctl: sysctl net.inet.ip.fw.nat64_debug=1 sysctl net.inet.ip.fw.verbose=1 See /var/log/security for output. ----- With these conditions the following tests were successful: [root@v6only ~]# ping6 -c 3 64:ff9b::203.0.113.10 PING6(56=40+8+8 bytes) 2001:db8:12::30 --> 64:ff9b::cb00:710a 16 bytes from 64:ff9b::cb00:710a, icmp_seq=0 hlim=63 time=8.401 ms 16 bytes from 64:ff9b::cb00:710a, icmp_seq=1 hlim=63 time=3.429 ms 16 bytes from 64:ff9b::cb00:710a, icmp_seq=2 hlim=63 time=3.398 ms --- 64:ff9b::203.0.113.10 ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 3.398/5.076/8.401/2.351 ms And using lynx to grab the nginx home page was successful: lynx external1.example.com ------- Welcome to nginx! Welcome to nginx! If you see this page, the nginx web server is successfully installed and working. Further configuration is required. This is Machine 1 - 203.0.113.10 For online documentation and support please refer to nginx.org. Commercial support is available at nginx.com. Thank you for using nginx. Are you sure you want to quit? (y) Arrow keys: Up and Down to move. Right to follow a link; Left to go back. ------- I am closing the ticket with the caveat that nat64lsn under 13.0 may still need fixing (identical source and destination MAC addresses in the reply). Closed : Works as Intended <--- but only in 13.1 Jim B. |
Created attachment 233913 [details] nat64lsn and dns64 test bed - 4 VMs on FreeBSD host See the attached figure (nat64_issue.png) for address layout. The FreeBSD host (my laptop), v6only VM, firewall VM, and the DNShost VM are all 13.0-RELEASE-p11. The external1 host (IPV4 only host) is 13.0-RELEASE-p8. This nat64lsn (stateful NAT64) example follows the BSD Router Project address layout and ruleset. However, these are stock FreeBSD 13.0 VMs (qemu-system-x86_64 version 6.2.0), not BSDRP images. TCP session request from IPv6 only host to IPv4 only host *almost* works. The initial SYN packet is NAT64ed correctly and reaches the destination IPv4 host who sends a SYN/ACK back. The ipfw instance on the router moves the SYN/ACK packet back through the ruleset and writes it out the proper interface (em1). But - the packet sent back to the IPV6 host on that interface has a malformed destination MAC address. ipfw_nat64 duplicated the em1 interface MAC address (the source) in the destination field: Wireshark trace shows SYN/ACK reply packet has duplicated source and destination MAC addresses: Ethernet II, Src: 02:49:50:46:57:42 (02:49:50:46:57:42), Dst: 02:49:50:46:57:42 (02:49:50:46:57:42) Destination: 02:49:50:46:57:42 (02:49:50:46:57:42) Address: 02:49:50:46:57:42 (02:49:50:46:57:42) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 02:49:50:46:57:42 (02:49:50:46:57:42) Address: 02:49:50:46:57:42 (02:49:50:46:57:42) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv6 (0x86dd) IPV6 host config: oot@v6only:~ # ifconfig em0 em0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=481209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP> ether 02:49:de:ad:be:ef inet6 2001:db8:12::1 prefixlen 64 inet6 fe80::49:deff:fead:beef%em0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ipfw router config: root@firewall:~ # ifconfig -a em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=481209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP> ether 02:49:50:46:57:41 inet 2.2.2.2 netmask 0xffffff00 broadcast 2.2.2.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> em1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=481209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP> ether 02:49:50:46:57:42 inet6 2001:db8:12::2 prefixlen 64 inet6 fe80::49:50ff:fe46:5742%em1 prefixlen 64 scopeid 0x2 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> root@firewall:~ # root@firewall:~ # ndp -a Neighbor Linklayer Address Netif Expire S Flags v6only.example.com 02:49:de:ad:be:ef em1 23h8m28s S 2001:db8:12::2 02:49:50:46:57:42 em1 permanent R fe80::49:50ff:fe46:5742%em1 02:49:50:46:57:42 em1 permanent R dnshost.example.com 02:49:53:53:54:54 em1 22h49m2s S root@firewall:~ # ipfw setup: root@firewall:~ # ipfw nat64lsn NAT64 show config nat64lsn NAT64 prefix4 2.2.1.0/24 prefix6 64:ff9b::/96 log root@firewall:~ # root@firewall:~ # ipfw show 00100 12 816 allow log ipv6-icmp from any to any icmp6types 135,136 00200 7 512 nat64lsn NAT64 log ip from 2001:db8:12::/64 to 64:ff9b::/96 in 00300 16 912 nat64lsn NAT64 log ip from any to 2.2.1.0/24 in 00400 58 5920 allow log ip from any to any 00500 0 0 allow log ip6 from any to any 65535 0 0 deny ip from any to any startup script: root@firewall:~ # cat bsdrp.sh #!/bin/sh set -x kldunload ipfw_nat64 kldunload ipfw sleep 1 kldload ipfw kldload ipfw_nat64 # Logging: 0 interfaces, 1 syslog sysctl net.inet.ip.fw.verbose=1 # Debug nat64 sysctl net.inet.ip.fw.nat64_debug=1 fwcmd="/sbin/ipfw" ${fwcmd} -f flush ${fwcmd} nat64lsn NAT64 create log prefix4 2.2.1.0/24 ${fwcmd} add allow log icmp6 from any to any icmp6types 135,136 ${fwcmd} add nat64lsn NAT64 log ip from 2001:db8:12::/64 to 64:ff9b::/96 in ${fwcmd} add nat64lsn NAT64 log ip from any to 2.2.1.0/24 in ${fwcmd} add allow log ip from any to any ${fwcmd} add allow log ip6 from any to any # Direct output: 1 enable, 0 disable (packet goes back into ruleset) sysctl net.inet.ip.fw.nat64_direct_output=1 Note that I've been running two dozen or more different ipfw tests using this same testbed and I have not encountered a similar issue with MAC addresses. I will double check all this when 13.1 lands.