I've got two free virtual machines in Oracle Cloud to run a VPN service there. I've noticed that the network speed was ok with direct connection, but was extreamly slow, when traffic crosses vtnet and VPN interfaces (tried both IPsec and OpenVPN). Adding hw.vtnet.csum_disable=1 fixed the problem. Seems like it is for a while in the wild, and still not fixed. https://serverfault.com/questions/817664/slow-network-fixed-adding-hw-vtnet-csum-disable-1-what-are-the-posible-side-effe
Currently freebsd-version shows 13.2-RELEASE-p9
Also the driver provides some hw.vtnet.fixup_needs_csum that is not documented in man page.
I think I have related experience with csum vtnet bug. On one VM (in KVM) with 2 vtnet NICs IPsec VPN the packets are not going thru until I disabled csum on "LAN" NIC vtnet1: ifconfig_vtnet0="inet AA.BB.CC.DD netmask 255.255.255.128" ifconfig_vtnet1="inet WW.XX.YY.ZZ/24 -rxcsum -txcsum" This machine works like an IPsec gateway, vtnet0 is facing the internet, vtnet1 connects private network with clients connecting thru IPsec. Without -rxcsum -txcsum on vtnet1 the client machines cannot connect to anything in the outside world. First discovered with FreeBSD 12.3 (installed in the VM), later upgraded to 13.2.
(In reply to crypt47 from comment #0) > but was extreamly slow, when traffic crosses vtnet and VPN interfaces May you please elaborate your setup that how the traffic crosses vtnet and VPN interfaces ?
Created attachment 248169 [details] port 80 dump
(In reply to Zhenlei Huang from comment #4) It is very simple. We have one machine "A" with outgoing interface (vtnet0) and some virtual (like ipsec0 or tun0). We also have another VM "B" in the same network and also vtnet0 interface which has nginx serving FreeBSD ISO file.:) If we directly request the ISO file from "A" works alight. If we request the file via VPN interfaces, we get a lot of retransmissions because packets began to be discarded. I'm attaching the dump file, captured on vtnet0.
If you view the attached 80.dump file with Wireshark and check "validate the TCP checksum" option, it will complain about server (10.0.0.53) packets. Can't say how much it's relevant, because it's not the server with VPN setup. It may be some Wireshark miscalculation.
As for the ipfw configuration, I don't want to post it in full, but I use ipfw_f add 10 reass all from any to any in // ASSEMBLING in the beginning. Some nat translation later. Nothing special. Also mss fix is used for IPsec, but it can't be relevant. Breaks with OpenVPN too.