Bug 240320 - ixgbe: EEE state change causes core dump on X552
Summary: ixgbe: EEE state change causes core dump on X552
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: Any Any
: --- Affects Many People
Assignee: Eric Joyner
URL:
Keywords: IntelNetworking, crash
Depends on:
Blocks: 240700
  Show dependency treegraph
 
Reported: 2019-09-03 20:54 UTC by Zach Vargas
Modified: 2019-10-16 21:55 UTC (History)
3 users (show)

See Also:
erj: mfc-stable11+
erj: mfc-stable12+


Attachments
Patch to disable EEE support on X552 devices in the ix driver. (504 bytes, patch)
2019-09-03 20:54 UTC, Zach Vargas
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Zach Vargas 2019-09-03 20:54:01 UTC
Created attachment 207168 [details]
Patch to disable EEE support on X552 devices in the ix driver.

Overview:

My system has a Xeon-D 1500 series SoC with two 10GbE Intel X552 backplane connections (device ID: 0x15AB). In FreeBSD 12.0 RELEASE, attempting to use the sysctl utility to transition Energy Efficient Ethernet (EEE) states causes a core dump (see below for details). I believe that this is the result of the fact that X552 devices do not support EEE. This assertion is based on the following documentation from Intel:

The note below table 1.2 (Network Features) in Section 1.6 of volume 4 of the datasheet for Xeon-D 1500 (https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xeon-d-1500-datasheet-vol-4.pdf) states:

 EEE is ONLY supported for 10GBASE-T interfaces via the X557-AT2 PHY. The Intel® Xeon® Processor D-1500 Product Family LANcontroller does not support EEE feature.

Furthermore, in the README for the Linux ixgbe driver (version 5.6.1), EEE is listed as an unsupported feature:

 NOTE: Devices based on the Intel(R) Ethernet Connection X552 and Intel(R)
 Ethernet Connection X553 do not support the following features:
 * Energy Efficient Ethernet (EEE)
 * Intel PROSet for Windows Device Manager
 * Intel ANS teams or VLANs (LBFO is supported)
 * Fibre Channel over Ethernet (FCoE)
 * Data Center Bridging (DCB)
 * IPSec Offloading
 * MACSec Offloading

The FreeBSD ix driver (version 3.3.10) release README also has a list of unsupported features for X552. However, these lists do not match. The FreeBSD version lists 'Low Latency Interrupts (LLI)' as the only unsupported feature.

I have attached a patch (0001-X552-does-not-support-eee.patch) to disable EEE support on X552 devices in the ix driver. This patch was created against r350420 of FreeBSD which was the latest in CURRENT at the time of my investigation.

Steps to Reproduce:

Attempt to change EEE state via the ix driver eee_state attribute via the sysctl utility.

Actual results:

Running 12.0 RELEASE FreeBSD from https://svn.freebsd.org/base/releng/12.0, attempting to change the EEE state of an ix adapter:

 root@bsd:~ # uname -r
 12.0-RELEASE
 root@bsd:~ # sysctl dev.ix.1.eee_state
 dev.ix.1.eee_state: 1
 root@bsd:~ # sysctl dev.ix.1.eee_state=0


 Fatal trap 12: page fault while in kernel mode
 cpuid = 5; apic id = 05
 fault virtual address  = 0x0
 fault code             = supervisor read instruction, page not present
 instruction pointer    = 0x20:0x0
 stack pointer          = 0x28:0xfffffe008a86d718
 frame pointer          = 0x28:0xfffffe008a86d760
 code segment           = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
 processor eflags       = interrupt enabled, resume, IOPL = 0
 current process                = 2424 (sysctl)
 trap number            = 12
 panic: page fault
 cpuid = 5
 time = 1563820937
 KDB: stack backtrace:
 #0 0xffffffff80bdcbc7 at kdb_backtrace+0x67
 #1 0xffffffff80b907b3 at vpanic+0x1a3
 #2 0xffffffff80b90603 at panic+0x43
 #3 0xffffffff8106996f at trap_fatal+0x35f
 #4 0xffffffff810699c9 at trap_pfault+0x49
 #5 0xffffffff81068fee at trap+0x29e
 #6 0xffffffff81044425 at calltrap+0x8
 #7 0xffffffff80b9f56b at sysctl_root_handler_locked+0x8b
 #8 0xffffffff80b9ec27 at sysctl_root+0x257
 #9 0xffffffff80b9f29a at userland_sysctl+0x17a
 #10 0xffffffff80b9f0df at sys___sysctl+0x5f
 #11 0xffffffff8106a449 at amd64_syscall+0x369
 #12 0xffffffff81044d0d at fast_syscall_common+0x101
 Uptime: 36m40s
 Dumping 736 out of 16329 MB:..3%..11%..22%..31%..42%..53%..61%..72%..81%..92%
 Dump complete

I see the same behaviour in CURRENT and core dump on boot in previous versions (11.3).

Expected Results: Transition EEE state or error out without coredump.

Testing with this patch applied:
 root@bsd:~/freebsd-ixgbe # sysctl dev.ix.1.eee_state
 sysctl: unknown oid 'dev.ix.1.eee_state'
 root@bsd:~/freebsd-ixgbe #
Comment 1 commit-hook freebsd_committer 2019-10-15 21:57:15 UTC
A commit references this bug:

Author: erj
Date: Tue Oct 15 21:56:20 UTC 2019
New revision: 353599
URL: https://svnweb.freebsd.org/changeset/base/353599

Log:
  ixgbe: Disable EEE for backplane X550EM_X

  From Zach:
  Intel documentation indicates that backplane X550EM_X KR devices do not
  support Energy Efficient Ethernet. Prior to this patch, X552 devices
  (device ID 0x15AB) will crash the system when transitioning EEE state
  via sysctl.

  Signed-off-by: Zach Vargas <zvargas@xes-inc.com>

  PR:		240320
  Submitted by:	Zach Vargas <zvargas@xes-inc.com>
  Reviewed by:	erj@
  MFC after:	3 days
  Differential Revision:	https://reviews.freebsd.org/D21673

Changes:
  head/sys/dev/ixgbe/if_ix.c
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-16 01:17:50 UTC
(In reply to commit-hook from comment #1)

@Eric, If this is a 12.1-R candidate, please add bug 240700 to this issues "Blocks" field
Comment 3 commit-hook freebsd_committer 2019-10-16 21:46:59 UTC
A commit references this bug:

Author: erj
Date: Wed Oct 16 21:46:49 UTC 2019
New revision: 353665
URL: https://svnweb.freebsd.org/changeset/base/353665

Log:
  MFC r353660: ixgbe: Disable EEE for backplane X550EM_X

  This prevents a kernel panic when configuring EEE on X552 devices.

  PR:		240320
  Approved by:	re@ (gjb@)
  Sponsored by:	Intel Corporation

Changes:
_U  releng/12.1/
  releng/12.1/sys/dev/ixgbe/if_ix.c
Comment 4 commit-hook freebsd_committer 2019-10-16 21:55:03 UTC
A commit references this bug:

Author: erj
Date: Wed Oct 16 21:54:49 UTC 2019
New revision: 353669
URL: https://svnweb.freebsd.org/changeset/base/353669

Log:
  MFC r353599

  MFCs ixgbe: Disable EEE for backplane X550EM_X

  This prevents a possible kernel panic on X552 backplane devices.

  PR:		240320
  Sponsored by:   Intel Corporation

Changes:
_U  stable/11/
  stable/11/sys/dev/ixgbe/if_ix.c