Hello, i am using Opnsense (freeBSD 13.0 stable) and i am trying to span a VXLAN over a wireguard (kernel-mod and/or go implementation) tunnel. wireguard as well as VXLAN can establish a connection but no data can be sent over the VXLAN tunnel. the first analysis shows that the first ARP request from the remote VXLAN interface is not answered. if i use for example a tinc tunnel instead of the wireguard tunnel or span the VXLAN directly from interface to interface everything works as expected. can anyone confirm this problem? is it possibly an opnsense specific problem? in a clean ubuntu 20.04 the desired setup works without problems. thank you very much and best regards, volker
How did you configure Wireguard's crypto key routing and did you configure VXLAN to use broadcast, multicast or unicast?
okay, sorry for the delay but now i have tested with an plain freeBSD 13-release (no opnsense offset). same result, VXLAN over wireguard isnt working now. here is my test environment config: system1: - network interface vtnet0 is in DHCP mode for internet uplink and ssh access - network interface vtnet1 is a dedicated network between the VMs (10GBit with 9000 MTU): # ifconfig vtnet1 inet 10.0.0.1 netmask 255.255.255.0 mtu 9000 # pkg install wireguard wireguard config: /usr/local/etc/wireguard/wg0.conf ============================================================= [Interface] PrivateKey = ODK5skFPghxdo41XWG9Way6fIKHfhvKxeMDVxkeE610= Address = 10.10.0.1/24 ListenPort = 51820 [Peer] PublicKey = 4jyrPvDJUPAe/Lvqa6XUuqDX1SzzMiFVADh56jSPAWU= AllowedIPs = 10.10.0.2/32 Endpoint = 10.0.0.2:51820 PersistentKeepalive = 15 ============================================================= # service wireguard start # ifconfig vxlan create vxlanid 42 vxlanlocal 10.10.0.1 vxlanremote 10.10.0.2 inet 10.10.10.1/24 system 2: - network interface vtnet0 is in DHCP mode for internet uplink and ssh access - network interface vtnet1 is a dedicated network between the VMs (10GBit with 9000 MTU): # ifconfig vtnet1 inet 10.0.0.2 netmask 255.255.255.0 mtu 9000 # pkg install wireguard wireguard config: /usr/local/etc/wireguard/wg0.conf ============================================================= [Interface] PrivateKey = ODK5skFPghxdo41XWG9Way6fIKHfhvKxeMDVxkeE610= Address = 10.10.0.2/24 ListenPort = 51820 [Peer] PublicKey = 4jyrPvDJUPAe/Lvqa6XUuqDX1SzzMiFVADh56jSPAWU= AllowedIPs = 10.10.0.1/32 Endpoint = 10.0.0.1:51820 PersistentKeepalive = 15 ============================================================= # service wireguard start # ifconfig vxlan create vxlanid 42 vxlanlocal 10.10.0.2 vxlanremote 10.10.0.1 inet 10.10.10.2/24 so all interfaces are up and running, so from system 1 i start to ping the interfaces: root@freebsd:~ # ping 10.0.0.2 PING 10.0.0.2 (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=64 time=0.581 ms 64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.614 ms ... so ping between the native interfaces are fine root@freebsd:~ # ping 10.10.0.2 PING 10.10.0.2 (10.10.0.2): 56 data bytes 64 bytes from 10.10.0.2: icmp_seq=0 ttl=64 time=0.824 ms 64 bytes from 10.10.0.2: icmp_seq=1 ttl=64 time=0.879 ms ... ping via wireguard are also fine root@freebsd:~ # ping 10.10.10.2 PING 10.10.10.2 (10.10.10.2): 56 data bytes ping: sendto: Host is down ping: sendto: Host is down ... ping over the VXLAN is not working :( an check to the arp-table shows the problem: root@freebsd:~ # arp -a ? (10.10.10.2) at (incomplete) on vxlan0 expired [ethernet] ... the initial arp request isnt responded by the opposite VXLAN interface. here is the ifconfig from system 1: root@freebsd:~ # ifconfig vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether 4a:78:ef:c5:db:28 inet6 fe80::4878:efff:fec5:db28%vtnet0 prefixlen 64 scopeid 0x1 inet 10.50.0.240 netmask 0xffffff00 broadcast 10.50.0.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> vtnet1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether c6:77:27:a1:fd:3e inet6 fe80::c477:27ff:fea1:fd3e%vtnet1 prefixlen 64 scopeid 0x3 inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> vxlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80020<JUMBO_MTU,LINKSTATE> ether 58:9c:fc:10:ff:c5 inet 10.10.10.1 netmask 0xffffff00 broadcast 10.10.10.255 inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 groups: vxlan vxlan vni 42 local 10.10.0.1:4789 remote 10.10.0.2:4789 media: Ethernet autoselect (autoselect <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 8920 options=80000<LINKSTATE> inet 10.10.0.1 netmask 0xffffff00 groups: wg nd6 options=103<PERFORMNUD,ACCEPT_RTADV,NO_DAD> and here from system 2: root@freebsd:/usr/local/etc/wireguard # ifconfig vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether 2e:c0:6b:ff:3b:79 inet6 fe80::2cc0:6bff:feff:3b79%vtnet0 prefixlen 64 scopeid 0x1 inet 10.50.0.241 netmask 0xffffff00 broadcast 10.50.0.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> vtnet1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether 52:10:64:d2:31:bf inet6 fe80::5010:64ff:fed2:31bf%vtnet1 prefixlen 64 scopeid 0x2 inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420 options=80000<LINKSTATE> inet 10.10.0.2 netmask 0xffffff00 groups: wg nd6 options=103<PERFORMNUD,ACCEPT_RTADV,NO_DAD> vxlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80020<JUMBO_MTU,LINKSTATE> ether 58:9c:fc:10:ff:c5 inet 10.10.10.2 netmask 0xffffff00 broadcast 10.10.10.255 inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 groups: vxlan vxlan vni 42 local 10.10.0.2:4789 remote 10.10.0.1:4789 media: Ethernet autoselect (autoselect <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> so, this my input, maybe someone can have a look and can help to solve this problem??? i know, the VXLAN MTU value isnt optimized but this shouldnt the problem...
i have add an fix and define the MTU value for both wireguard interfaces to 1420 and VXLAN to 1370. but nothing changed...
More and more users are attaching to this issue, has anyone an hint to this problem? is it wireguard itself, i do not think that there is an different between the other distro (e.g. debian/ubuntu/etc) versions? or is it an bsd problem? maybe we can collect some meenings around the this issue to bring more attractive to this thread. thx.
Re-assign to Ports given the issue is reproducible with it. If a base or base networking issue is isolated to be a root cause, please create a separate dependent issue for resolution.
@Reporter Please confirm the package version of wireguard-kmod (pkg info) and full uname -a of the (FreeBSD) system it was reproduced on
Hi there, I have the same issue. Hopefully my submission of uname -a and pkg info wireguard-kmod will help this bug progress further. I'm sure if this bug gets resolved, it will also resolve my issue. For completeness my setup is this: - 2x 13.1-RELEASE-p2 host - no firewalls active - Working wireguard setup (can ssh from one to the other via the wireguard IPs) - Working vxlan over ipsec setup, traffic across the vxlan tunnel ends up on the other side. In my case the vxlan interfaces are bridged to a real interface so that I end up having a stretched ethernet network. When I change the vxlanlocal and vxlanremote on both ends from the ipsec ip addresses to the wireguard ip adresses the traffic across the vxlan tunnel does not end up on the other side any more. I do see the udp/4789 packets arrive on the wg0 interface, but they don't seem to make it into the vxlan interface. My suspicion is that by the time the packets get decrypted by the wireguard setup, it's too late to have them get processed by vxlan. This is pure speculation. My `uname -a` is this: (identical on both ends) FreeBSD <hostname> 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC amd64 my `pkg info wireguard-kmod` is this: wireguard-kmod-0.0.20220615 Name : wireguard-kmod Version : 0.0.20220615 Installed on : Fri Sep 16 11:29:06 2022 CEST Origin : net/wireguard-kmod Architecture : FreeBSD:13:amd64 Prefix : /usr/local Categories : net-vpn net kld Licenses : MIT Maintainer : decke@FreeBSD.org WWW : https://git.zx2c4.com/wireguard-freebsd/ Comment : WireGuard implementation for the FreeBSD kernel Annotations : FreeBSD_version: 1301000 repo_type : binary repository : FreeBSD Flat size : 144KiB Description : Kernel module for FreeBSD to support Wireguard. At this time this code is new, unvetted, possibly buggy, and should be considered "experimental". It might contain security issues. We gladly welcome your testing and bug reports, but do keep in mind that this code is new, so some caution should be exercised at the moment for using it in mission critical environments. WWW: https://git.zx2c4.com/wireguard-freebsd/
Hi, I had same issue, vxlan traffic not passed thruogh wireguard kmod on FreeBSD 12.2 and 13.0 STABLE I have found that the FreeBSD's wireguard kernel implementation is the problem itself. I removed Kernel implementation and installed go user space implementation it worked on FreeBSD 12.2 STABLE bot tried for 13.0. Now i tried vxlan over wg on FreebSD 14 CURRENT 20220923 AMD64 build this time both wireguard kmod & go implementation didn't work. Configuration: wireguard freebsd 14 machine wg0 interface ip :192.168.199.1/24 wireguard linux machine wg0 iface ip : 192.168.199.250/24 wireguard peers connected each other successfully and available to ping one by one but when i added vxlan like this : ifconfig vxlan700 create vxlanid 700 vxlanlocal 192.168.199.1 vxlanremote 192.168.199.250 ifconfig vxlan700 192.168.88.1/24 and i configured linux machine to meet this vxlan700 there was no traffic enters vxlan700 on bsd side The traffic doesn't pass from wireguard to vxlan when i listened wg interface "udp port vxlan unreachable" error occurs here is FreeBSD side logs: peer: >secret< endpoint: >secret<:10626 allowed ips: 192.168.199.250/32 latest handshake: Now transfer: 145.32 KiB received, 181.92 KiB sent persistent keepalive: every 25 seconds root@freebsd14:~ # tcpdump -i vxlan700 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vxlan700, link-type EN10MB (Ethernet), capture size 262144 bytes >>> there is no traffic root@freebsd14:~ # tcpdump -i wg0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on wg0, link-type NULL (BSD loopback), capture size 262144 bytes 17:21:11.801802 IP 192.168.199.250.47705 > 192.168.199.1.vxlan: VXLAN, flags [I] (0x08), vni 700 ARP, Request who-has 192.168.88.1 tell 192.168.88.2, length 46 17:21:11.801832 IP 192.168.199.1 > 192.168.199.250: ICMP 192.168.199.1 udp port vxlan unreachable, length 104 17:21:12.825053 IP 192.168.199.250.47705 > 192.168.199.1.vxlan: VXLAN, flags [I] (0x08), vni 700 ARP, Request who-has 192.168.88.1 tell 192.168.88.2, length 46 There was same problem for freebsd 12.2 and 13 stable versions wg kmod implementations i have created a topic shown below . I was solved installing go implementation but this not worked for freebsd 14 . https://forums.freebsd.org/threads/vxlan-works-only-on-freebsd-11-2-but-not-works-on-12-2-and-13-stable.81833/#post-566850 i used latest freebsd 14 current image https://download.freebsd.org/snapshots/amd64/amd64/ISO-IMAGES/14.0/FreeBSD-14.0-CURRENT-amd64-20220923-00d8a28f19b-258155-memstick.img.xz
I managed to repeat this issue on FreeBSD 13.1 with wireguard-kmod-0.0.20220615 . There are some weird kernel logs looks like this: ``` Sep 30 00:00:42 kernel: vxlan0: discard frame w/o leading ethernet header (len 8 pkt len 42) ... ``` It seems that the mbuf is not pullup before passing to the ethernet stack.
(In reply to Zhenlei Huang from comment #9) Could you try shown below? . Worked successfully for me on FreeBSD 12.2 Stable Wireguard-kmod and vxlan not works together i experienced. pkg remove wireguard-kmod pkg remove wireguard-tools pkg install wireguard-go pkg install wireguard-tools kldunload if_wg wg-quick down wg0 wg-quick up wg0 i think it will work for you if not please share tcpdump -i wg0 and tcpdump -i vxlan700
Created attachment 236944 [details] Draft patch vxlan(4) vxlan_input() requires at least ETHER_HDR_LEN of mbuf.
(In reply to Alfa from comment #10) I've test the patch on stable/13 and it works excellently. Can you please test patch on stable/12 ?
See also: https://reviews.freebsd.org/D36794
(In reply to Zhenlei Huang from comment #12) i am currently working on FreeBSD 14.0 CURRENT 20220923 build. Now i can test it on FreeBSD 14.0 later test it 12/stable I removed wg-go implementation and reinstall wg-kmod then i am also getting below kernel messages: Sep 29 17:21:59 test kernel: vxlan700: discard frame w/o leading ethernet header (len 8 pkt len 163) Sep 29 17:22:00 test kernel: vxlan700: discard frame w/o leading ethernet header (len 8 pkt len 856)
(In reply to Alfa from comment #14) (In reply to Zhenlei Huang from comment #12) i have tested your patch (attachment 236944 [details]) on FreeBSD 14 Current but not worked. I am still getting kernel: vxlan700: discard frame w/o leading ethernet header (len 8 pkt len 856) But when i removed wg-kmod and install wg-go pkg remove wireguard-kmod pkg remove wireguard-tools pkg install wireguard-go pkg install wireguard-tools and then changed vxlan port 4789 to 5000 on both devices then it worked successfully.
(In reply to Alfa from comment #15) [UPDATE] I reapplied your patch to FreeBSD 14.0 Current 20220923 and it worked perfectly thanks for your work. Now wireguard-kmod and vxlan works as expected.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=8707cb19e68128c181477d87f960182c14e15f12 commit 8707cb19e68128c181477d87f960182c14e15f12 Author: Zhenlei Huang <zlei.huang@gmail.com> AuthorDate: 2022-09-30 07:55:29 +0000 Commit: Kristof Provost <kp@FreeBSD.org> CommitDate: 2022-09-30 07:56:15 +0000 if_vxlan(4): Check the size of data available in mbuf before using them PR: 261711 Reviewed by: kp Differential Revision: https://reviews.freebsd.org/D36794 sys/net/if_vxlan.c | 6 ++++++ 1 file changed, 6 insertions(+)
(In reply to commit-hook from comment #17) Triage: assign to committer resolving. Someone in IRC asks whether this bug can be closed …
(In reply to Graham Perrin from comment #18) You'd have to ask the reporter. I've never reproduced this bug. I committed an obvious improvement to vxlan but do not have the setup with (or interest in) fireguard.
(In reply to Kristof Provost from comment #19) Thanks for clarifying. Assignment reverted to ports-bugs@
^Triage: To progress, this issue needs confirmation of resolution (or explicit confirmation it doesn't) for the issue as originally reported. @Reporter (Volker, anyone else with issue reproduction) please test a stable/13 kernel, or 13.x-RELEASE kernel with patch applied and report if/if not resolved. (In reply to Kristof Provost from comment #19) @Kristof Original reports are for (at least) stable/13, is src 8707cb19e681 (comment 17) MFC'able or has it already been, and is it needed for stable/12 too, or not applicable? ^Triage: Reset status (no assignee)
@G(In reply to Graham Perrin from comment #21) > ^Triage: To progress, this issue needs confirmation of resolution (or explicit > confirmation it doesn't) for the issue as originally reported. > @Reporter (Volker, anyone else with issue reproduction) please test a stable/13 > kernel, or 13.x-RELEASE kernel with patch applied and report if/if not resolved. > (In reply to Kristof Provost from comment #19) > @Kristof Original reports are for (at least) stable/13, is src 8707cb19e681 (comment 17) MFC'able or has it already been, and is it needed for stable/12 too, or not applicable? I thinks it also applies to stable/12 , from the source code aspect. The routine triggers this bug is same on stable/12, stable/13 and CURRENT. And @Alfa (not the original reporter) also reported the issue on 12.2, see comment #8
(In reply to Zhenlei Huang from comment #22) I tried to apply patch to stable/12.2 and it gives me an error. FreeBSD test 12.2-STABLE FreeBSD 12.2-STABLE r369958 GENERIC amd64 root@test:/usr/src/sys/net # patch -i if_vxlan_mbuf_patch.patch Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- b/sys/net/if_vxlan.c |+++ b/sys/net/if_vxlan.c -------------------------- Patching file if_vxlan.c using Plan A... Hunk #1 failed at 1. 1 out of 1 hunks failed--saving rejects to if_vxlan.c.rej done root@test:/usr/src/sys/net #
Created attachment 237750 [details] stable/12 patch (In reply to Alfa from comment #23) There're some changes of if_vxlan not MFCed to stable/12, thus causing conflicts. Can you please try this patch `stable/12 patch` for stable/12 ?
(In reply to Zhenlei Huang from comment #24) I successfully patched and builded new kernel but We have a problem with wireguard-kmod in stable/12 . >> root@test:~ # wg-quick up wg0 [#] ifconfig wg create name wg0 [!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation. [#] wireguard-go wg0 /usr/local/bin/wg-quick: line 33: wireguard-go: command not found [#] ifconfig wg0 destroy ifconfig: interface wg0 does not exist for couple of months i didn't experience this problem. There must be some shanges on wireguard root@test:~ # kldload if_wg kldload: an error occurred while loading module if_wg. Please check dmesg(8) for more details. KLD if_wg.ko: depends on kernel - not available or version mismatch linker_load_file: /boot/modules/if_wg.ko - unsupported file type FreeBSD test 12.2-STABLE FreeBSD 12.2-STABLE ALFA amd64
(In reply to Alfa from comment #25) > root@test:~ # kldload if_wg > kldload: an error occurred while loading module if_wg. Please check dmesg(8) for more details. > KLD if_wg.ko: depends on kernel - not available or version mismatch > linker_load_file: /boot/modules/if_wg.ko - unsupported file type It seems the kernel is too old. > FreeBSD test 12.2-STABLE FreeBSD 12.2-STABLE ALFA amd64 It looks weird. The kernel version should be 12.4-STABLE if you compile src code from stable/12 branch and install the kernel. Is your stable/12 branch up to date ? PS: While we say try something on stable / current, it implicitly means we should try latest stable / current, unless with explicit git revision.
(In reply to Alfa from comment #25) And also please check the if_wg.ko module is not installed from 13.x pkg site. You can verify by `pkg info wireguard-kmod`. Beware the `Architecture` and `FreeBSD_version` .
(In reply to Zhenlei Huang from comment #26) Ok you are right . I cloned stable/12 from git just now and patched vxlan so now i have: FreeBSD test 12.4-STABLE FreeBSD 12.4-STABLE #0 stable/12-n235817-fb7823f55ce7-dirty: Fri Nov 4 10:12:06 +03 2022 Then i set up the scenerio it seems it works! Thanks for your work. FreeBSD test 12.4-STABLE FreeBSD 12.4-STABLE #0 stable/12-n235817-fb7823f55ce7-dirty: Fri Nov 4 10:12:06 +03 2022 root@test:/usr/obj/usr/src/amd64.amd64/sys/TEST amd64 root@testbsd:~ # tcpdump -i wg0 -n tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on wg0, link-type NULL (BSD loopback), capture size 262144 bytes 12:19:00.505479 IP 192.168.222.2.36750 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.52648 > ff02::c.3702: UDP, length 607 12:19:00.513287 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:01.418336 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:01.508249 IP 192.168.222.2.50345 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::2: ICMP6, router solicitation, length 16 12:19:02.505552 IP 192.168.222.2.58966 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.42705 > ff02::c.3702: UDP, length 607 12:19:02.505566 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:03.308397 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:03.405801 IP 192.168.222.2.44032 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.32816 > ff02::c.3702: UDP, length 607 12:19:03.405805 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:04.398449 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 clinent ▶ sudo tcpdump -i wg0 -n tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on wg0, link-type RAW (Raw IP), snapshot length 262144 bytes 12:19:02.438382 IP 192.168.222.2.36750 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.52648 > ff02::c.3702: UDP, length 607 12:19:02.438422 IP 192.168.222.2.51623 > 239.255.255.250.3702: UDP, length 607 12:19:02.451364 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:02.451413 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:02.655331 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:03.351531 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:03.447359 IP 192.168.222.2.50345 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::2: ICMP6, router solicitation, length 16 12:19:04.429377 IP 192.168.222.2.58966 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.42705 > ff02::c.3702: UDP, length 607 12:19:04.429418 IP 192.168.222.2.44135 > 239.255.255.250.3702: UDP, length 607 12:19:04.443458 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:04.443505 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:05.115291 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:05.243344 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:05.304430 IP 192.168.222.2.44032 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b.32816 > ff02::c.3702: UDP, length 607 12:19:05.304463 IP 192.168.222.2.40178 > 239.255.255.250.3702: UDP, length 607 12:19:05.319385 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:05.319449 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:19:05.687360 IP 192.168.222.2 > 224.0.0.22: igmp v3 report, 1 group record(s) 12:19:06.327384 IP 192.168.222.2.38627 > 192.168.222.1.4789: VXLAN, flags [I] (0x08), vni 700 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 root@testbsd:~ # tcpdump -i vxlan700 -n tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vxlan700, link-type EN10MB (Ethernet), capture size 262144 bytes 12:21:07.085249 IP6 fe80::a48d:a9ff:fe2e:51b.42409 > ff02::c.3702: UDP, length 607 12:21:07.085257 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:21:08.027610 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:21:08.174671 IP6 fe80::a48d:a9ff:fe2e:51b.45469 > ff02::c.3702: UDP, length 607 12:21:08.174677 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:21:09.004628 IP6 fe80::a48d:a9ff:fe2e:51b.34016 > ff02::c.3702: UDP, length 607 12:21:09.012281 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:21:09.247722 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 client ▶ sudo tcpdump -i vxlan700 -n tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on vxlan700, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:20:24.543573 IP6 fe80::a48d:a9ff:fe2e:51b.35890 > ff02::c.3702: UDP, length 607 12:20:24.555415 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:20:24.679469 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:20:25.963918 IP6 fe80::a48d:a9ff:fe2e:51b.47275 > ff02::c.3702: UDP, length 607 12:20:25.979498 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:20:26.487418 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:20:26.822411 IP6 fe80::a48d:a9ff:fe2e:51b.56986 > ff02::c.3702: UDP, length 607 12:20:26.835608 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 12:20:27.575516 IP6 fe80::a48d:a9ff:fe2e:51b > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
(In reply to Alfa from comment #28) That's good news !
(In reply to Zhenlei Huang from comment #29) > That's good news ! @Kristof Will this catch up the release window ? 12.4 goes to RC1 recently. It would be bad news if not.
(In reply to Zhenlei Huang from comment #30) We're pretty late in the cycle for more patches, but it's sufficiently small and self-contained that we might be able to persuade re@ to take it. We do need to get it into stable/12 (and 13) asap so we can ask. Did you want to prepare the MFC, or should I do them to get them in asap?
(In reply to Kristof Provost from comment #31) > We're pretty late in the cycle for more patches, but it's sufficiently small and > self-contained that we might be able to persuade re@ to take it. That's good ! > We do need to get it into stable/12 (and 13) asap so we can ask. > Did you want to prepare the MFC, or should I do them to get them in asap? I'd glad to if approved :)
(In reply to Zhenlei Huang from comment #32) Remember to 'git cherry-pick -x' so there's a `(cherry picked from commit 8707cb19e68128c181477d87f960182c14e15f12)` line, and you'll have to git commit --amend to add the 'Approved by: kp (mentor)' line. Other than that, go for it.
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0156c3f1f7608bb1ce547bed6b60cb084db3e7eb commit 0156c3f1f7608bb1ce547bed6b60cb084db3e7eb Author: Zhenlei Huang <zlei.huang@gmail.com> AuthorDate: 2022-09-30 07:55:29 +0000 Commit: Zhenlei Huang <zlei@FreeBSD.org> CommitDate: 2022-11-04 10:38:50 +0000 if_vxlan(4): Check the size of data available in mbuf before using them PR: 261711 Reviewed by: kp Tested by: Alfa (burak.sn at outlook.com) Approved by: kp (mentor) Differential Revision: https://reviews.freebsd.org/D36794 (cherry picked from commit 8707cb19e68128c181477d87f960182c14e15f12) sys/net/if_vxlan.c | 6 ++++++ 1 file changed, 6 insertions(+)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=639aebb2aa0b72c569f8779e13351d37f7bb17c6 commit 639aebb2aa0b72c569f8779e13351d37f7bb17c6 Author: Zhenlei Huang <zlei.huang@gmail.com> AuthorDate: 2022-09-30 07:55:29 +0000 Commit: Zhenlei Huang <zlei@FreeBSD.org> CommitDate: 2022-11-04 10:57:23 +0000 if_vxlan(4): Check the size of data available in mbuf before using them PR: 261711 Reviewed by: kp Tested by: Alfa (burak.sn at outlook.com) Approved by: kp (mentor) Differential Revision: https://reviews.freebsd.org/D36794 (cherry picked from commit 8707cb19e68128c181477d87f960182c14e15f12) sys/net/if_vxlan.c | 6 ++++++ 1 file changed, 6 insertions(+)
A commit in branch releng/12.4 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=a9c3d5de9ac8ae8090881f20c894d3577d97e92a commit a9c3d5de9ac8ae8090881f20c894d3577d97e92a Author: Zhenlei Huang <zlei.huang@gmail.com> AuthorDate: 2022-09-30 07:55:29 +0000 Commit: Kristof Provost <kp@FreeBSD.org> CommitDate: 2022-11-04 12:21:45 +0000 if_vxlan(4): Check the size of data available in mbuf before using them PR: 261711 Reviewed by: kp Tested by: Alfa (burak.sn at outlook.com) Approved by: re (gjb), kp (mentor) Differential Revision: https://reviews.freebsd.org/D36794 (cherry picked from commit 8707cb19e68128c181477d87f960182c14e15f12) (cherry picked from commit 639aebb2aa0b72c569f8779e13351d37f7bb17c6) sys/net/if_vxlan.c | 6 ++++++ 1 file changed, 6 insertions(+)