Summary: | FreeBSD 12.0 ix driver TX TSO vlan-tagged network regression from 11.2 | ||
---|---|---|---|
Product: | Base System | Reporter: | Nick <freebsd> |
Component: | misc | Assignee: | freebsd-net (Nobody) <net> |
Status: | New --- | ||
Severity: | Affects Only Me | Keywords: | regression |
Priority: | --- | ||
Version: | 12.0-RELEASE | ||
Hardware: | amd64 | ||
OS: | Any |
Description
Nick
2019-01-05 00:06:25 UTC
Hi, I've done some additional testing: -Failing over to the igb interfaces doesn't show any problems with TSO enabled or disabled, so this is probably specific to ix -Dropping the MTU to 1500 has no effect -Removing an ix interfaces from its lagg and configuring an untagged interface directly on it doesn't shows significantly but improved TX performance, however this is seen with TSO enabled and disabled. I haven't managed to rule out the switch configuration as the cause of this yet, as I don't normally have any untagged ports. I'll update in a bit when I've done some more testing with the untagged interface configuration. Kind regards, Nick (In reply to Nick from comment #1) Sorry, I snafu'd that last comment. Should have read: -Removing an ix interface from its lagg and configuring an untagged interface directly on it shows significantly improved, but still poor, TX performance - however this is seen both with TSO enabled and disabled. I haven't managed to rule out the switch configuration as the cause of this yet, as I don't normally have any untagged ports. Hi, One further point of data: ipv6 testing shows the same results as ipv4, both for tagged and untagged interfaces. I've also discovered something odd: once TSO has been disabled on an interface, or if it was disabled at boot, I'm unable to subsequently trigger the issue on a tagged interface. Either turning TSO on after boot for some reason doesn't cause the problem, or my attempts to turn TSO on after boot are failing. I'm not really sure how to check for definite, but I do seee that the tsx_tx counters don't increase from 0 in this case, so I suspect TSO hasn't really been re-enabled for some reason: Booted with TSO disabled on ix0: 19:09:27 root@filer:/root/ sysctl dev.ix.0 | grep -i tso dev.ix.0.queue3.tso_tx: 0 dev.ix.0.queue2.tso_tx: 0 dev.ix.0.queue1.tso_tx: 0 dev.ix.0.queue0.tso_tx: 0 Enable TSO: 19:15:11 root@filer:/root/ ifconfig ix0 tso TX test shows good behaviour: 19:18:00 root@filer:/root/ iperf3 -c 10.21.10.4 Connecting to host 10.21.10.4, port 5201 [ 5] local 10.21.10.2 port 54437 connected to 10.21.10.4 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.15 GBytes 9.90 Gbits/sec 0 1.09 MBytes [ 5] 1.00-2.00 sec 1.15 GBytes 9.88 Gbits/sec 0 1.09 MBytes [ 5] 2.00-3.00 sec 1.15 GBytes 9.89 Gbits/sec 0 1.09 MBytes [ 5] 3.00-4.00 sec 1.15 GBytes 9.90 Gbits/sec 0 1.09 MBytes [ 5] 4.00-5.00 sec 1.15 GBytes 9.90 Gbits/sec 0 1.09 MBytes [ 5] 5.00-6.00 sec 1.15 GBytes 9.89 Gbits/sec 0 1.09 MBytes [ 5] 6.00-7.00 sec 1.15 GBytes 9.89 Gbits/sec 0 1.09 MBytes [ 5] 7.00-8.00 sec 1.15 GBytes 9.88 Gbits/sec 0 1.09 MBytes [ 5] 8.00-9.00 sec 1.15 GBytes 9.90 Gbits/sec 0 1.09 MBytes [ 5] 9.00-10.00 sec 1.15 GBytes 9.90 Gbits/sec 0 1.09 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 11.5 GBytes 9.89 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 11.5 GBytes 9.89 Gbits/sec receiver Re-check TSO counters, discover they haven't increased: 19:18:11 root@filer:/root/ sysctl dev.ix.0 | grep -i tso dev.ix.0.queue3.tso_tx: 0 dev.ix.0.queue2.tso_tx: 0 dev.ix.0.queue1.tso_tx: 0 dev.ix.0.queue0.tso_tx: 0 I'm pretty much out of ideas as to how to proceeed with diagnosis at this point. I'm happy to perform any further testing you may need, but disabling TSO seems to adequately solve the problem for me, so I'll just do that. If you want to attempt to reproduce, I think our minimal config to reproduce is thus: FreeBSD 12.0-Release (amd64) running with an X520-DA2 card installed, with one port in a failover lagg, configured with a vlan-tagged interface on top, with TSO enabled on ix<whichever> in rc.conf. In case the platform is relevant, this machine is a Xeon E3-1245v3 (with HT disabled in BIOS), running on an ASRock C226WS with 4 x DDR3 ECC DIMMs installed. Kind regards, Nick |