Bug 236067 - [em] Intel 82574L can not turn off flow control
Summary: [em] Intel 82574L can not turn off flow control
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net mailing list
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2019-02-26 22:21 UTC by Lev A. Serebryakov
Modified: 2019-03-02 13:07 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lev A. Serebryakov freebsd_committer 2019-02-26 22:21:23 UTC
I have two systems with 82574L NICs.

One, name it G (for "generator"), is powerful enough to saturate almost 1G links with small packets (more than 1100Kpps).

Other, name it DUT is underpowered and can not process such packet rate.

Both G and dut have "set dev.em.0.fc=0".

G try to generate traffic to DUT's MAC with netmap & pkt-gen. If G and DUT are connected (no matter how: with direct patchcord or via dedicated switch) G could generate only as much packets as DUT could process! If I remove DUT from dedicated switch, G generates almost-line-rate PPS. If I replace DUT with other system with I210 NIC, G generates same high PPS even if system with I210 can not process it.

Looks like 82574L can not turn off flow control.

Once again:

82574L -> Switch -> 82574L - pkt-gen generates PPS limited by receiver.
82574L -> 82574L - pkt-gen generates PPS limited by receiver.
82574L -> Switch -> "blackhole" (no other devices attached to switch) - pkt-gen generates line-rate PPS.
82574L -> I210 - pkt-gen generates line-rate PPS.
82574L -> Switch -> I210 - pkt-gen generates line-rate PPS.
Comment 1 Eric Joyner freebsd_committer 2019-02-27 22:12:51 UTC
Are you using the in-kernel driver in CURRENT on both sides? Do they both have 82574L?
Comment 2 Eric Joyner freebsd_committer 2019-02-27 22:15:00 UTC
(In reply to Eric Joyner from comment #1)

Sorry, I re-read your initial comment; it looks like this problem only occurs when both sides are 82574L. That's interesting.
Comment 3 Lev A. Serebryakov freebsd_committer 2019-02-28 11:18:12 UTC
(In reply to Eric Joyner from comment #2)
Both sides are r344277, drivers are compiled in kernel (not loaded as modules).

And I didn't tried "i210 -> 82574L" yet, I could try this combination around Monday.
Comment 4 Lev A. Serebryakov freebsd_committer 2019-03-02 12:42:07 UTC
(In reply to Eric Joyner from comment #2)
I've added i210 (in form of I210-T1 addon card) to my traffic generator system, and it is limited by receiver (82574L) too!

Both ends have FC turned off (with `dev.igb.0.fc=0` and `dev.em.0.fc=0` respective). I've tried with and without switch between. So, two more datapoints:

i210 -> Switch -> 82574L - pkt-gen generates PPS limited by receiver.
i210 -> 82574L - pkt-gen generates PPS limited by receiver.
i210 -> Switch -> "blackhole" (no other devices attached to switch) - pkt-gen generates line-rate PPS.
Comment 5 Lev A. Serebryakov freebsd_committer 2019-03-02 13:07:43 UTC
Inserting manageable switch (HP1800-8G) instead of simple one (HP1400-8G) with explicitly disabled flow control on all ports helps, BTW.
But still, i210->i210 doesn't require manageable switch to drop packets on the floor :-)