| Summary: | virtio vtnet driver with rxcsum errors | ||
|---|---|---|---|
| Product: | Base System | Reporter: | mike |
| Component: | kern | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | New --- | ||
| Severity: | Affects Only Me | CC: | emaste, vikashb, void |
| Priority: | --- | ||
| Version: | 13.1-STABLE | ||
| Hardware: | Any | ||
| OS: | Any | ||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165059 | ||
Actually this looks related to this long standing bug :( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165059 |
I was setting up a VM pf firewall and noticed I was not able to nat out for some reason. Looking at the pcap, it seems when the vm is in forwarding mode, I get tcp checksum errors. If I do a ifconfig vtnet1 -rxcsum ifconfig vtnet0 -rxcsum nat then seems to work fine The setup is a simple VM with the hypervisor libvirt/KVM ubuntu 20 LTS. Guest is RELENG_13 from Apr 11/2022 e.g. vtnet1 is facing another internal VM. If I try and connect to the internal NIC vtnet1 (192.168.199.7) I dont get any errors. But if I try and get out via nat, it fails. If I switch both nics to Intel em NICs, it works without modification public_internet <---> vtnet0-vtnet1 <---> another guest vm vtnet0 = public IP vtnet1 = 192.168.199.7/24 guest = 192.168.199.100 tcpdump -s0 -vnei vtnet1 port 443 tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes 18:35:05.937364 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 64547, offset 0, flags [DF], proto TCP (6), length 60) 192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect -> 0x4755), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val 1606656477 ecr 0,nop,wscale 7], length 0 18:35:06.939305 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 64548, offset 0, flags [DF], proto TCP (6), length 60) 192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect -> 0x436b), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val 1606657479 ecr 0,nop,wscale 7], length 0 18:35:21.040936 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 45298, offset 0, flags [DF], proto TCP (6), length 60) 192.168.199.100.59876 > 192.168.199.7.443: Flags [S], cksum 0x0fec (incorrect -> 0xf8f6), seq 1122263205, win 64240, options [mss 1460,sackOK,TS val 2124691085 ecr 0,nop,wscale 7], length 0 18:35:21.040993 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 192.168.199.7.443 > 192.168.199.100.59876: Flags [S.], cksum 0x0fec (incorrect -> 0x5be5), seq 3948792593, ack 1122263206, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2270714410 ecr 2124691085], length 0 18:35:21.041224 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 45299, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x0fe4 (incorrect -> 0x88b9), ack 1, win 502, options [nop,nop,TS val 2124691086 ecr 2270714410], length 0 18:35:22.843871 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 45300, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.100.59876 > 192.168.199.7.443: Flags [F.], cksum 0x0fe4 (incorrect -> 0x81ae), seq 1, ack 1, win 502, options [nop,nop,TS val 2124692888 ecr 2270714410], length 0 18:35:22.843910 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.7.443 > 192.168.199.100.59876: Flags [.], cksum 0x0fe4 (incorrect -> 0x7895), ack 2, win 1027, options [nop,nop,TS val 2270716214 ecr 2124692888], length 0 18:35:22.843980 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.7.443 > 192.168.199.100.59876: Flags [F.], cksum 0x0fe4 (incorrect -> 0x7894), seq 1, ack 2, win 1027, options [nop,nop,TS val 2270716214 ecr 2124692888], length 0 18:35:22.844159 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x7aa0 (correct), ack 2, win 502, options [nop,nop,TS val 2124692889 ecr 2270716214], length 0 e.g. after disabling rxcsum (ifconfig vtnet1 -rxcsum ifconfig vtnet0 -rxcsum) tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes 18:39:57.425940 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 53398, offset 0, flags [DF], proto TCP (6), length 60) 192.168.199.101.55810 > 142.251.32.68.443: Flags [S], cksum 0x5467 (correct), seq 2136333436, win 64240, options [mss 1460,sackOK,TS val 2746738194 ecr 0,nop,wscale 7], length 0 18:39:57.432020 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 123, id 58394, offset 0, flags [none], proto TCP (6), length 60) 142.251.32.68.443 > 192.168.199.101.55810: Flags [S.], cksum 0xa929 (correct), seq 292800283, ack 2136333437, win 65535, options [mss 1430,sackOK,TS val 460829236 ecr 2746738194,nop,wscale 8], length 0 18:39:57.432265 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53399, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xd5dc (correct), ack 1, win 502, options [nop,nop,TS val 2746738200 ecr 460829236], length 0 18:39:58.605990 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53400, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.101.55810 > 142.251.32.68.443: Flags [F.], cksum 0xd145 (correct), seq 1, ack 1, win 502, options [nop,nop,TS val 2746739374 ecr 460829236], length 0 18:39:58.612128 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 123, id 58984, offset 0, flags [none], proto TCP (6), length 52) 142.251.32.68.443 > 192.168.199.101.55810: Flags [F.], cksum 0xcd9e (correct), seq 1, ack 2, win 256, options [nop,nop,TS val 460830416 ecr 2746739374], length 0 18:39:58.612402 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53401, offset 0, flags [DF], proto TCP (6), length 52) 192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xcca2 (correct), ack 2, win 502, options [nop,nop,TS val 2746739380 ecr 460830416], length 0 I am guessing txcsum is fine as I can connect out from the FreeBSD VM, so packets originating on the firewall's vtnet0 interface is ok.