Router: FreeBSD 13 Network behind NAT: some Linux hosts Router: --------------------------------------- [root@balancer1:/etc]# ifconfig | grep -A1 vtnet | grep options options=4c079b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> options=4c079b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> Host behind NAT, several curl launches: --------------------------------------- [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 527k 0 0 150k 0 17:55:26 0:00:03 17:55:23 150k^C [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 671k 0 0 188k 0 14:15:29 0:00:03 14:15:26 188k^C [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 1599k 0 0 198k 0 13:34:49 0:00:08 13:34:41 214k^C [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 1647k 0 0 225k 0 11:56:44 0:00:07 11:56:37 218k^C [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 1247k 0 0 134k 0 20:05:17 0:00:09 20:05:08 147k^C [root@back1 server]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 383k 0 0 185k 0 14:31:44 0:00:02 14:31:42 185k^C Router: ---------------------------------------------- [root@balancer1:/etc]# ifconfig vtnet0 -rxcsum [root@balancer1:/etc]# ifconfig vtnet1 -rxcsum [root@balancer1:/etc]# ifconfig vtnet1 -txcsum [root@balancer1:/etc]# ifconfig vtnet0 -txcsum Same host behind NAT: ---------------------------------------------- [root@back1 etc]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 3 9469M 3 309M 0 0 89.6M 0 0:01:45 0:00:03 0:01:42 89.6M^C [root@back1 etc]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 9469M 0 76.9M 0 0 85.2M 0 0:01:51 --:--:-- 0:01:51 85.2M^C [root@back1 etc]# curl --output /dev/null https://mirror.yandex.ru/centos/8.4.2105/isos/x86_64/CentOS-8.4.2105-x86_64-dvd1.iso % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 1 9469M 1 99.3M 0 0 123M 0 0:01:16 --:--:-- 0:01:16 123M^C Guess it's self-explanatory.
Are you sure this is a regression? I do not know of a single version of FreeBSD that works without turning off [rx|tx]csum and LRO.
Well... not sure at this moment. May be not a regression, but ...is there a bug for this ? BTW: Workaround: it's enough to only turn off the rxcsum [and LRO ?]. With txcsum also turned off the ssh session became unresponsive (though receiving works just fine).
Can you retest with stable/14, stable/15, releng/15 or current?
As fast as as I have testing stand (see #276760), I've re-measured in NATed jail and here clear degradation of download for 14.3p4 with offloading if compare with 14-stable or no offloading ~26 Mbps vs ~800 Mbps normally FreeBSD Offloading Download (CV%) delta % Upload (CV%) delta % 14.3p4 yes 26 19.96% 445 19.96% 14.3p4 no 813 23.93% 3079% 528 23.93% 19% 14-087fc6ae0a8c yes 489 58.95% -40% 431 58.95% -18% 14-087fc6ae0a8c no 762 33.78% 56% 570 33.78% 32% raw data: FreeBSD Offloading Try Download Upload 14.3p4 yes 1 21 54 14.3p4 yes 2 18 769 14.3p4 yes 3 20 315 14.3p4 yes 4 28 540 14.3p4 yes 5 33 495 14.3p4 yes 6 27 356 14.3p4 yes 7 32 421 14.3p4 yes 8 22 325 14.3p4 yes 9 28 451 14.3p4 yes 10 27 728 14.3p4 yes avg 26 445 CV% 19.96% 46.73% 14.3p4 no 1 932 686 14.3p4 no 2 749 433 14.3p4 no 3 854 441 14.3p4 no 4 885 624 14.3p4 no 5 946 760 14.3p4 no 6 284 211 14.3p4 no 7 916 754 14.3p4 no 8 863 462 14.3p4 no 9 883 416 14.3p4 no 10 816 496 14.3p4 no avg 813 528 CV% 23.93% 33.02% 14-087fc6ae0a8c yes 1 87 510 14-087fc6ae0a8c yes 2 537 763 14-087fc6ae0a8c yes 3 884 576 14-087fc6ae0a8c yes 4 714 414 14-087fc6ae0a8c yes 5 379 392 14-087fc6ae0a8c yes 6 724 441 14-087fc6ae0a8c yes 7 259 563 14-087fc6ae0a8c yes 8 758 474 14-087fc6ae0a8c yes 9 60 48 14-087fc6ae0a8c yes 10 484 128 14-087fc6ae0a8c yes avg 489 431 CV% 58.95% 48.74% 14-087fc6ae0a8c no 1 964 541 14-087fc6ae0a8c no 2 943 410 14-087fc6ae0a8c no 3 980 478 14-087fc6ae0a8c no 4 981 761 14-087fc6ae0a8c no 5 861 757 14-087fc6ae0a8c no 6 874 537 14-087fc6ae0a8c no 7 765 471 14-087fc6ae0a8c no 8 596 510 14-087fc6ae0a8c no 9 327 511 14-087fc6ae0a8c no 10 328 728 14-087fc6ae0a8c no avg 762 570 CV% 33.78% 22.59%
(In reply to vova from comment #4) Could you guide me (possibly via private email) to configure your setup on one of my local machines? I would like to be able to do such measurements (upload/download) in a controlled environment to see what effects are related to vtnet and its interaction with nat on the one side and on changes in the network on the other side.
(In reply to Michael Tuexen from comment #5) I've sent details in private email
(In reply to Michael Tuexen from comment #5) VM Setup instructions roughly: install FreeBSD 14.3 from ISO image, ZFS-based (amd64 machine - CX23 on hetzner) freebsd-update fetch && freebsd-update install pkg update pkg install speedtest-go cbsd /usr/local/cbsd/sudoexec/initenv /usr/local/cbsd/share/initenv.conf default_vs=1 workdir=/jails # CBSD will create basic pf nat rules and bridge interface - cbsdbr0 with 10.0.0.1/24 cbsd jcreate jname=speedtest vnet=1 interface=cbsdbr0 sysrc=defaultrouter=10.0.0.1 astart=1 cbsd jstart speedtest cbsd jlogin speedtest jail# pkg install speedtest-go jail# speedtest-go --server 60469