Bug 258133 - toggling TSO fixes latency/slowness
Summary: toggling TSO fixes latency/slowness
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-08-29 08:12 UTC by nbari
Modified: 2022-07-25 13:10 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description nbari 2021-08-29 08:12:01 UTC
The main interface in the system is igb0, it has a vlan0 interface with other gateway (fib 1) in where in theory TSO is enable by default:

vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1400
        options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>


I started to notice some latency issues on the VLAN and by just enabling again TSO the problem was fixed:

ifconfig vlan0 -tso 
ifconfig vlan0 tso


Is there a way to ensure that TSO is always enabled also for newly created interfaces or probably by enabling/disabling TSO fixes other issues?
Comment 1 Daniel Ponte 2022-02-08 01:46:19 UTC
I am not certain if my issue is related, but I am noticing issues with the same setup (13-STABLE, igb(4) interface has vlan interfaces created, but issue is present on default vlan). My machine is an nginx reverse proxy to a grafana server (also a FreeBSD machine) inside. From boot, with TSO enabled, I get weird timeouts and retransmits. Turning off or toggling TSO resolves them. I have determined that the issue is not present with a 13-RELEASE kernel. As the specific payloads causing my issue have secrets in them, I unfortunately cannot post them here, but would be happy to work with someone privately.

13.0-STABLE FreeBSD 13.0-STABLE #1 stable/13-n249226-be7f063b764d: Thu Feb  3 13:05:25 EST 2022     root@argon.h.c907:/usr/obj/usr/src/amd64.amd64/sys/FARWARL  amd64
Comment 3 Hans Petter Selasky freebsd_committer freebsd_triage 2022-02-08 15:18:54 UTC
The affect LRO and not TSO, but may be related.
Comment 4 Hans Petter Selasky freebsd_committer freebsd_triage 2022-02-08 15:19:33 UTC
Do other nics, like Mellanox ones, exhibit the same behaviour?
Comment 5 Daniel Ponte 2022-02-08 19:53:04 UTC
(In reply to Hans Petter Selasky from comment #2)

These patches make no difference, unfortunately. I only have Intel NICs on my FreeBSD machines, so I am unable to test.
Comment 6 Daniel Ponte 2022-02-08 20:04:20 UTC
(In reply to Daniel Ponte from comment #5)

To clarify, unable to test with other NICs. I have tested these patches with an Intel NIC and they made no difference.
Comment 7 Aleksandr Fedorov freebsd_committer freebsd_triage 2022-02-13 13:32:57 UTC
(In reply to Daniel Ponte from comment #6)
May you show ifconfig output for vlan and Intel interface?
Comment 8 Daniel Ponte 2022-02-13 13:40:37 UTC
(In reply to Aleksandr Fedorov from comment #7)

igb1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 00:e0:67:18:5d:09
        inet 10.68.80.1 netmask 0xffffff00 broadcast 10.68.80.255
        inet6 fe80::2e0:67ff:fe18:5d09%igb1 prefixlen 64 scopeid 0x2
        inet6 2001:XYZXYZXYZ::3 prefixlen 64
        inet6 2001:XYZXYZXYZ::3 prefixlen 64
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 00:e0:67:18:5d:09
        inet 10.68.81.1 netmask 0xffffff00 broadcast 10.68.81.255
        inet6 fe80::2e0:67ff:fe18:5d09%vlan0 prefixlen 64 scopeid 0xb
        groups: vlan
        vlan: 132 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
iot0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 00:e0:67:18:5d:09
        inet 10.68.83.1 netmask 0xffffff00 broadcast 10.68.83.255
        inet6 fe80::2e0:67ff:fe18:5d09%iot0 prefixlen 64 scopeid 0xc
        groups: vlan
        vlan: 133 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Comment 9 Daniel Ponte 2022-07-25 13:10:03 UTC
Just an update, this continues to occur on 13.1-STABLE #5 stable/13-n251918-7914eea8e65c: Mon Jul 25 02:29:48 EDT 2022. I can furnish pcaps on request, but I won't publish them here as they contain semiprivate information.