Bug 234570

Summary: em(4) 12.0 regressions: Does not function after resume, hw.em.{txd,rxd} ignored
Product: Base System Reporter: Marek Zarychta <zarychtam>
Component: kernAssignee: freebsd-net (Nobody) <net>
Status: Closed FIXED    
Severity: Affects Only Me CC: erj
Priority: --- Keywords: IntelNetworking, needs-qa, performance, regression
Version: 12.0-STABLEFlags: koobs: mfc-stable12?
Hardware: amd64   
OS: Any   

Description Marek Zarychta 2019-01-02 14:07:49 UTC
I have recently upgraded one box running 11.2-STABLE to 12.0-STABLE, but unexpectedly occurred some issues regarding em(4) NIC.

Wake on LAN still works, but NIC is deaf after resume regardless the machine was woken up by NIC itself or ACPI button. After resume the switch reports 1000 link, but when the port is cycled down/up the link doesn't bring up any more. No problems are reported in the kernel message buffer.

The second issue comes from constant number of TX/RX queues. Both loader tunables: hw.em.txd and hw.em.rxd are silently ignored and there are always "em0: netmap queues/slots: TX 1/1024, RX 1/1024"

Also, the opportunity to enable "options EM_MULTIQUEUE" in the kernel config file has been depreciated which seems like planned degradation of performance for this popular NIC.

The affected computer has this network card:

em0@pci0:0:25:0:	class=0x020000 card=0x1905103c chip=0x153a8086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I217-LM'
    class      = network
    subclass   = ethernet
    cap 01[c8] = powerspec 2  supports D0 D3  current D0
    cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
    cap 13[e0] = PCI Advanced Features: FLR TP

Suspend/resume and wake on LAN worked fine in 11.2-STABLE, not counting the fact that the machine was able to suspended only once. Suspending after resume worked, but workstation was waking up almost immediately (with "sysctl dev.em.0.wake=1" set). Next long lasting sleep state was possible only after reboot.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2019-01-02 14:18:37 UTC
Thank you for the report Marek

Might be worth testing the driver from net/intel-em-kmod, if nothing else than to establish/identify potential functional deltas between what base has and the upstream driver.
Comment 2 Eric Joyner freebsd_committer freebsd_triage 2019-01-02 17:58:41 UTC
(In reply to Marek Zarychta from comment #0)

Hi; I can respond to a couple of these that aren't WoL-related:

In 12, the em(4) driver uses the iflib framework, so the tunables have changed. To change the number of descriptors in the queues, use the "dev.em.0.iflib.override_ntxds" and "dev.em.0.iflib.override_nrxds" sysctls. These are marked as tunables, so you can also set these in /boot/loader.conf, and iflib should use these values when the driver is loaded.

For "options EM_MULTIQUEUE", it appears that only really changed the driver for the 82574 adapter in 11, so that option wouldn't have applied to your i217 adapter. Regardless, that functionality has been turned on by default for MSI-X capable em cards (which 82574 is the only one so far), so the option isn't needed anymore.
Comment 3 Marek Zarychta 2019-01-02 18:45:05 UTC
(In reply to Eric Joyner from comment #2)
Thank you for the explanation and clarification. I must admit that "dev.em.0.iflib.override_ntxds" and "dev.em.0.iflib.override_nrxds" tunables work as "hw.em.rxd" and "hw.em.txd" used to. Indeed, this I217-LM wasn't using multiple queues while running 11-STABLE, but I build the same  kernel for a few boxes with different em(4) NICs. Likely only one of them seems to be multiqueue capable.
Comment 4 Marek Zarychta 2019-01-15 06:26:12 UTC
I am closing this because the most critical issue reported here was a duplicate o the bug 224059 which has been fixed. Knobs "dev.em.0.iflib.override_ntxds" and "dev.em.0.iflib.override_nrxds" superseding "hw.em.rxd" and "hw.em.txd" are still not properly documented in em(4) manual page, but this is a minor issue only.