Bug 280390 - NPTv6 not working
Summary: NPTv6 not working
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.3-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ipfw (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-07-21 13:58 UTC by cnbatch
Modified: 2024-07-25 17:22 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description cnbatch 2024-07-21 13:58:25 UTC
https://forums.freebsd.org/threads/how-to-properly-configure-nptv6.88971/

This issue still exists in FreeBSD 13.3-RELEASE
Comment 1 Marek Zarychta 2024-07-21 14:35:32 UTC
(In reply to cnbatch from comment #0)
Please, provide the full test case to reproduce.
Comment 2 cnbatch 2024-07-21 14:53:38 UTC
(In reply to Marek Zarychta from comment #1)

Could you please specify what additional information or steps are needed?

I think I've already outlined the configuration and testing process in my previous thread post.
Comment 3 Andrey V. Elsukov freebsd_committer freebsd_triage 2024-07-22 07:32:28 UTC
(In reply to cnbatch from comment #0)
You need to make sure, that IPv6 packets are handled by NPT rule in both directions. But as I see, you use `allow ip6 from any to any via vtnet0` rule just before NPT rule. It will consume all packets that are going from external to internal network and it is why NPT does not work.
Comment 4 cnbatch 2024-07-22 16:53:56 UTC
Still not able to `ping6 freebsd.org` from client after changing the ipfw rules:

#!/bin/sh
ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd allow icmp6 from any to any
$cmd allow icmp from any to any
$cmd nptv6 NPT ip6 from any to any
$cmd allow ip6 from any to any via vtnet0
ipfw -q nat 1 config if vtnet0 same_ports unreg_only reset
$cmd nat 1 ip4 from any to any via vtnet0
$cmd allow all from any to any
$cmd check-state
Comment 5 John Hay 2024-07-22 17:55:44 UTC
ping6 use icmp6 and your rules allow icmp6 before you get to the nptv6 rule, so the packets won't get a chance to get translated back.
Comment 6 cnbatch 2024-07-22 19:21:58 UTC
Now I've changed the rules:

#!/bin/sh
ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd nptv6 NPT ip6 from any to any
$cmd allow icmp6 from any to any
$cmd allow icmp from any to any
$cmd allow ip6 from any to any via vtnet0
ipfw -q nat 1 config if vtnet0 same_ports unreg_only reset
$cmd nat 1 ip4 from any to any via vtnet0
$cmd allow all from any to any

Much worse than previous settings, I can't even `ping freebsd.org` nor `ping6 freebsd.org` on the server console.
Comment 7 John Hay 2024-07-23 17:34:09 UTC
Are you still doing this on vultr like you said on the forum thread? What IPv6 address do you get from them? Just a single address or a subnet? Keep in mind that nptv6 translates from subnet to another subnet of the same size, and do not touch the port numbers. It is not like IPv4 NAT that can translate a whole internal network to a single IPv4 address and adjusting port numbers to not clash.

My ISP hands out a /56 and I then use a /64 of that to translate an internal /64 to with nptv6.
Comment 8 cnbatch 2024-07-23 18:12:48 UTC
(In reply to John Hay from comment #7)

A subnet: 2a05:f480:1c00:XXXX::/64
Comment 9 John Hay 2024-07-23 18:35:38 UTC
(In reply to cnbatch from comment #8)

Do they route that whole subnet to you in addition to the address you received via SLAAC on vtnet0? Or is that the address you received via SLAAC on vtnet0? If it is your vtnet0 address, you only have one address or at least the kernel will think so.

What do you see if you do "tcpdump -i vtnet0 -n" while trying to ping an IPv6 address outside? Can you see the translated packet going out? Can you see something coming back?
Comment 10 cnbatch 2024-07-23 20:00:11 UTC
(In reply to John Hay from comment #9)

Looks like they route the whole subnet.

If I put `ifconfig_vtnet0_alias0="inet6 2a05:f480:1c00:XXXX::ABCD prefixlen 64" ` in rc.conf, and turn off firewall, I can ping this address from other computers (USA, UK, Austira)
Comment 11 cnbatch 2024-07-23 20:14:35 UTC
(In reply to John Hay from comment #9)

Then I turn on the firewall again, and run `tcpdump -i vtnet0 -n` on server when `ping6 freebsd.org` from wireguard client.

With the folowing configuration:

ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd allow ip6 from any to any via vtnet0
$cmd nptv6 NPT ip6 from any to any
ipfw -q nat 1 config if vtnet0 same_ports unreg_only reset
$cmd nat 1 ip4 from any to any via vtnet0
$cmd allow all from any to any
$cmd check-state

Packes captured:

19:57:36.964105 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:37.489100 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1170, seq 0, length 16
19:57:37.989427 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:38.497729 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1170, seq 1, length 16
19:57:39.013522 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:39.417340 IP6 2a05:f480:1c00:2c:8ef7::2.55923 > 2a03:f80:XXXX:552b::1.57557: UDP, length 74
19:57:39.417352 IP6 2a05:f480:1c00:2c:8ef7::2.37967 > 2a03:f80:XXXX:552b::1.59532: UDP, length 74
19:57:39.418139 IP6 2a05:f480:1c00:2c:8ef7::2.22101 > 2a03:f80:XXXX:552b::1.58384: UDP, length 74
19:57:39.418147 IP6 2a05:f480:1c00:2c:8ef7::2.27653 > 2a03:f80:XXXX:552b::1.59241: UDP, length 74
19:57:39.418276 IP6 2a05:f480:1c00:2c:8ef7::2.42824 > 2a03:f80:XXXX:552b::1.59432: UDP, length 74
19:57:39.507683 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1170, seq 2, length 16
19:57:40.394101 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:40.574897 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1170, seq 3, length 16
19:57:41.445433 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:42.469438 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:43.929069 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:44.965499 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:45.989433 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:47.101635 IP6 2a05:f480:1c00:2c:8ef7::2.55923 > 2a03:f80:XXXX:552b::1.57557: UDP, length 74
19:57:47.101644 IP6 2a05:f480:1c00:2c:8ef7::2.37967 > 2a03:f80:XXXX:552b::1.59532: UDP, length 74
19:57:47.101646 IP6 2a05:f480:1c00:2c:8ef7::2.22101 > 2a03:f80:XXXX:552b::1.58384: UDP, length 74
19:57:47.101649 IP6 2a05:f480:1c00:2c:8ef7::2.27653 > 2a03:f80:XXXX:552b::1.59241: UDP, length 74
19:57:47.101821 IP6 2a05:f480:1c00:2c:8ef7::2.42824 > 2a03:f80:XXXX:552b::1.59432: UDP, length 74
19:57:47.123314 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:48.175339 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:49.189405 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
19:57:51.609310 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
Comment 12 cnbatch 2024-07-23 20:15:04 UTC
With the configuration:

#!/bin/sh
ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd allow icmp6 from any to any
$cmd allow icmp from any to any
$cmd nptv6 NPT ip6 from any to any
$cmd allow ip6 from any to any via vtnet0
ipfw -q nat 1 config if vtnet0 same_ports unreg_only reset
$cmd nat 1 ip4 from any to any via vtnet0
$cmd allow all from any to any
$cmd check-state


caputured:

20:01:54.529756 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:54.529770 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:54.529777 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:54.530495 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:54.530532 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:54.829912 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:54.829928 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:54.829936 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:54.830099 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:54.830113 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:54.851953 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:01:55.280496 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:55.280506 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:55.280936 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:55.280941 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:55.280942 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:55.908572 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:01:55.954848 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:55.955296 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:55.955413 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:55.955418 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:55.955420 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:56.932540 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:01:56.967080 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:56.967092 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:56.967479 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:56.967486 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:56.967489 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:57.681664 IP6 fdc9:281f:4d7:9ee9::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1193, seq 0, length 16
20:01:58.485005 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:01:58.485782 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:01:58.485865 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:01:58.485870 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:01:58.485960 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:01:58.509630 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:01:58.686017 IP6 fdc9:281f:4d7:9ee9::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1193, seq 1, length 16
20:01:59.364586 IP6 fe80::fc00:5ff:fe07:578d > fe80::5400:5ff:fe07:578d: ICMP6, neighbor solicitation, who has fe80::5400:5ff:fe07:578d, length 32
20:01:59.364627 IP6 fe80::5400:5ff:fe07:578d > fe80::fc00:5ff:fe07:578d: ICMP6, neighbor advertisement, tgt is fe80::5400:5ff:fe07:578d, length 24
20:01:59.556555 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:01:59.710208 IP6 fdc9:281f:4d7:9ee9::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1193, seq 2, length 16
20:02:00.580563 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:00.735676 IP6 fdc9:281f:4d7:9ee9::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1193, seq 3, length 16
20:02:00.761636 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:02:00.761647 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:02:00.762086 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:02:00.762236 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:02:00.762276 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:02:03.576065 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:04.177409 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:02:04.177420 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:02:04.177422 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:02:04.177428 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:02:04.177430 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:02:04.613557 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:05.637566 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:06.838005 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:07.876555 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:08.900556 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:09.299695 IP6 2a05:f480:1c00:2c:8ef7::2.46975 > 2a03:f80:XXXX:552b::1.59281: UDP, length 74
20:02:09.299710 IP6 2a05:f480:1c00:2c:8ef7::2.33220 > 2a03:f80:XXXX:552b::1.57885: UDP, length 74
20:02:09.299714 IP6 2a05:f480:1c00:2c:8ef7::2.60415 > 2a03:f80:XXXX:552b::1.59283: UDP, length 74
20:02:09.299716 IP6 2a05:f480:1c00:2c:8ef7::2.21071 > 2a03:f80:XXXX:552b::1.58223: UDP, length 74
20:02:09.299718 IP6 2a05:f480:1c00:2c:8ef7::2.62420 > 2a03:f80:XXXX:552b::1.57961: UDP, length 74
20:02:10.274252 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:11.332731 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:02:12.356545 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
Comment 13 cnbatch 2024-07-23 20:15:27 UTC
With the configuration:

#!/bin/sh
ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd nptv6 NPT ip6 from any to any
$cmd allow icmp6 from any to any
$cmd allow icmp from any to any
$cmd allow ip6 from any to any via vtnet0
ipfw -q nat 1 config if vtnet0 same_ports unreg_only reset
$cmd nat 1 ip4 from any to any via vtnet0
$cmd allow all from any to any
$cmd check-state

captured:

20:10:52.345074 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:53.380628 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:54.265688 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:54.265698 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:54.266375 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:54.266383 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:54.266386 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:54.404592 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:54.466137 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:54.466151 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:54.466769 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:54.466846 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:54.466907 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:54.481396 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1205, seq 0, length 16
20:10:54.766084 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:54.766095 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:54.766765 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:54.767361 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:54.767371 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:55.216168 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:55.216178 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:55.216832 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:55.217422 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:55.217431 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:55.505021 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1205, seq 1, length 16
20:10:55.583522 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:55.892051 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:55.892063 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:55.892079 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:55.892085 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:55.892088 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:56.514639 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1205, seq 2, length 16
20:10:56.644610 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:56.902689 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:56.902701 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:56.903305 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:56.903370 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:56.903435 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:56.964610 IP6 fe80::fc00:5ff:fe07:578d > fe80::5400:5ff:fe07:578d: ICMP6, neighbor solicitation, who has fe80::5400:5ff:fe07:578d, length 32
20:10:56.964645 IP6 fe80::5400:5ff:fe07:578d > fe80::fc00:5ff:fe07:578d: ICMP6, neighbor advertisement, tgt is fe80::5400:5ff:fe07:578d, length 24
20:10:57.525159 IP6 2a05:f480:1c00:2c:8ef7::2 > 2610:1c1:1:606c::50:15: ICMP6, echo request, id 1205, seq 3, length 16
20:10:57.668609 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:58.421353 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:10:58.421368 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:10:58.422080 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:10:58.422095 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:10:58.422177 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:10:58.944557 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:10:59.972594 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:00.698378 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:11:00.698389 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:11:00.699086 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:11:00.699251 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:11:00.699256 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:11:00.996628 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:04.112853 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:11:04.112865 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:11:04.114562 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:11:04.114599 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:11:04.114607 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:11:04.135448 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:05.156656 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:06.180615 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:09.235162 IP6 2a05:f480:1c00:2c:8ef7::2.43131 > 2a03:f80:XXXX:552b::1.57042: UDP, length 74
20:11:09.235174 IP6 2a05:f480:1c00:2c:8ef7::2.63514 > 2a03:f80:XXXX:552b::1.59823: UDP, length 74
20:11:09.236840 IP6 2a05:f480:1c00:2c:8ef7::2.25972 > 2a03:f80:XXXX:552b::1.59274: UDP, length 74
20:11:09.236849 IP6 2a05:f480:1c00:2c:8ef7::2.12124 > 2a03:f80:XXXX:552b::1.58151: UDP, length 74
20:11:09.236990 IP6 2a05:f480:1c00:2c:8ef7::2.33932 > 2a03:f80:XXXX:552b::1.57280: UDP, length 74
20:11:09.256398 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
20:11:10.276608 IP6 fe80::fc00:5ff:fe07:578d > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a05:f480:1c00:2c:8ef7::2, length 32
Comment 14 John Hay 2024-07-24 14:48:53 UTC
Looking at your tcpdumps, one can see that the packets are correctly translated before being transmitted, so that part is working. The problem is that because the vtnet0 interface is an ethernet like interface, the kernel is only going to accept or respond to the addresses that are configured on the interface. That is why you see the router sending neighbor solicitation messages, but your vm does not respond to them. That is not a ipfw or nptv6 problem, but more a kernel networking thing.

If you can get another /64 from vultr that they will route to you, you can configure that on lo0 and then let nptv6 use that for the external prefix.

Where I have seen nptv6 work with a /64 ipv6 address that is configured on the external interface is with pppoe setups, but ppp and tun interfaces behave differently, they do not need neighbor solicitations, packets are just sent.

If you only have a few devices, you might be able to configure proxy ndp for their external addresses, but I have not tried that.
Comment 15 John Hay 2024-07-25 10:53:36 UTC
I have made a test setup with roughly your setup, but added the ndproxy port, and added this to my rc.conf:

<snip>
ndproxy_enable="YES"
ndproxy_uplink_interface="vtnet0"
# mac and link-local address of upstream router
ndproxy_downlink_mac_address="0c:07:42:82:00:01"
ndproxy_uplink_ipv6_addresses="fe80::e07:42ff:fe82:1"
</snip>

I tweaked the /etc/ipfw.rules a little and left the ipv4 stuff out:
<snip>
#!/bin/sh
ipfw -q -f flush
cmd="ipfw -q add "
ipfw disable one_pass
ipfw nptv6 NPT create int_prefix fdc9:281f:4d7:9ee9:: ext_if vtnet0 prefixlen 64
$cmd allow ip6 from fe80::/10 to ff02::/16
$cmd allow ip6 from fe80::/10 to fe80::/10
$cmd nptv6 NPT ip6 from any to any via vtnet0
$cmd allow icmp6 from any to any
$cmd allow icmp from any to any
$cmd allow ip6 from any to any
</snip>

With this I had a working ping6.
Comment 16 cnbatch 2024-07-25 16:21:42 UTC
(In reply to John Hay from comment #15)

thanks for your testing.

I fully copied these configurations and replaced with the mac address of vtnet0, ping6 failed on server and client.
replaced with the mac address of uplink router's, ping6 still not working on server and client.

I'm giving up.
Comment 17 John Hay 2024-07-25 16:55:41 UTC
Looking at a previous tcpdump, did you also change the link local to:

ndproxy_uplink_ipv6_addresses="fe80::fc00:5ff:fe07:578d"

and the mac address, I think, but not 100% sure about the first byte (fe), to:

ndproxy_downlink_mac_address="fe:00:05:07:57:8d"

If you are still willing to try, a "tcpdump -i vtnet0 -n -e", might be useful.
Comment 18 cnbatch 2024-07-25 17:22:44 UTC
(In reply to John Hay from comment #17)
Yes,

ndproxy_uplink_ipv6_addresses="fe80::fc00:5ff:fe07:578d"

and

ndproxy_downlink_mac_address="fe:00:05:07:57:8d"

are both set in rc.conf

When I use `ping6` from VPN client, `tcpdump` on server captured IPv4 packets only, no IPv6 packet is caputred.