The main interface in the system is igb0, it has a vlan0 interface with other gateway (fib 1) in where in theory TSO is enable by default: vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1400 options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> I started to notice some latency issues on the VLAN and by just enabling again TSO the problem was fixed: ifconfig vlan0 -tso ifconfig vlan0 tso Is there a way to ensure that TSO is always enabled also for newly created interfaces or probably by enabling/disabling TSO fixes other issues?
I am not certain if my issue is related, but I am noticing issues with the same setup (13-STABLE, igb(4) interface has vlan interfaces created, but issue is present on default vlan). My machine is an nginx reverse proxy to a grafana server (also a FreeBSD machine) inside. From boot, with TSO enabled, I get weird timeouts and retransmits. Turning off or toggling TSO resolves them. I have determined that the issue is not present with a 13-RELEASE kernel. As the specific payloads causing my issue have secrets in them, I unfortunately cannot post them here, but would be happy to work with someone privately. 13.0-STABLE FreeBSD 13.0-STABLE #1 stable/13-n249226-be7f063b764d: Thu Feb 3 13:05:25 EST 2022 root@argon.h.c907:/usr/obj/usr/src/amd64.amd64/sys/FARWARL amd64
Hi, I've just merged the four following patches from -current to 13-stable: https://cgit.freebsd.org/src/commit/?id=ca73c0e25d7f306b15657267d6d372a2e6b38948 https://cgit.freebsd.org/src/commit/?id=cac68fff9f0500f5417dfe5634f13ecf9fdf2eaa https://cgit.freebsd.org/src/commit/?id=22b24dc1abb3bc41c9b54054c2242b525daa5bfc https://cgit.freebsd.org/src/commit/?id=4287b7caa921131845837dd394581d0246f38eb1 Can you test? --HPS
The affect LRO and not TSO, but may be related.
Do other nics, like Mellanox ones, exhibit the same behaviour?
(In reply to Hans Petter Selasky from comment #2) These patches make no difference, unfortunately. I only have Intel NICs on my FreeBSD machines, so I am unable to test.
(In reply to Daniel Ponte from comment #5) To clarify, unable to test with other NICs. I have tested these patches with an Intel NIC and they made no difference.
(In reply to Daniel Ponte from comment #6) May you show ifconfig output for vlan and Intel interface?
(In reply to Aleksandr Fedorov from comment #7) igb1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether 00:e0:67:18:5d:09 inet 10.68.80.1 netmask 0xffffff00 broadcast 10.68.80.255 inet6 fe80::2e0:67ff:fe18:5d09%igb1 prefixlen 64 scopeid 0x2 inet6 2001:XYZXYZXYZ::3 prefixlen 64 inet6 2001:XYZXYZXYZ::3 prefixlen 64 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether 00:e0:67:18:5d:09 inet 10.68.81.1 netmask 0xffffff00 broadcast 10.68.81.255 inet6 fe80::2e0:67ff:fe18:5d09%vlan0 prefixlen 64 scopeid 0xb groups: vlan vlan: 132 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> iot0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP> ether 00:e0:67:18:5d:09 inet 10.68.83.1 netmask 0xffffff00 broadcast 10.68.83.255 inet6 fe80::2e0:67ff:fe18:5d09%iot0 prefixlen 64 scopeid 0xc groups: vlan vlan: 133 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Just an update, this continues to occur on 13.1-STABLE #5 stable/13-n251918-7914eea8e65c: Mon Jul 25 02:29:48 EDT 2022. I can furnish pcaps on request, but I won't publish them here as they contain semiprivate information.