Bug 236724 - igb(4): Interfaces fail to switch active to inactive state
Summary: igb(4): Interfaces fail to switch active to inactive state
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-net mailing list
URL:
Keywords: IntelNetworking, needs-qa, regression
Depends on:
Blocks:
 
Reported: 2019-03-22 21:50 UTC by ncrogers
Modified: 2019-06-20 21:04 UTC (History)
6 users (show)

See Also:
koobs: mfc-stable12?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ncrogers 2019-03-22 21:50:11 UTC
Since upgrading to 12.0-RELEASE / iflib I've noticed that my igbX interfaces
no longer go from status "active" to "inactive" (as reported by ifconfig)
when the Ethernet port is unplugged and loses link. The status lights on the
physical interface go out but ifconfig still reports active +
autoselect/1000baseT. An "ifconfig down" followed by "up", while its
unplugged, forces the link to recognize the "inactive" state. After the
down/up dance plugging the cable in again returns it to "active".

I have replicated this problem with a clean install of the latest 12-STABLE
snapshot as of today. The only configuration I've made was the usual
timezone, root password, etc. during install. The interfaces in this test
case have no IP address configurations.

This is possibly a duplicate of bug #228556, however I am unsure because the
description is confusing and I am not using a virtualization layer.

I've noticed this on multiple systems with the following Intel devices (as reported by pciconf "device"):

I210 Gigabit Network Connection
I211 Gigabit Network Connection
Ethernet Connection I354

uname -a:
FreeBSD test.local 12.0-STABLE FreeBSD 12.0-STABLE r345358 GENERIC  amd64

ifconfig when igb2 is actually connected and active:
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:90:0b:78:13:5a
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ifconfig when igb2 is disconnected (identical to above):
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:90:0b:78:13:5a
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Relevant pciconf -lcv:
igb2@pci0:0:20:0:       class=0x020000 card=0x1f418086 chip=0x1f418086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x2000]
    cap 10[a0] = PCI-Express 2 root endpoint max data 512(512) FLR NS
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 00900bffff78135a
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
igb3@pci0:0:20:1:       class=0x020000 card=0x1f418086 chip=0x1f418086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x2000]
    cap 10[a0] = PCI-Express 2 root endpoint max data 512(512) FLR NS
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 00900bffff78135a
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
igb4@pci0:0:20:2:       class=0x020000 card=0x1f418086 chip=0x1f418086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x2000]
    cap 10[a0] = PCI-Express 2 root endpoint max data 512(512) FLR NS
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 00900bffff78135a
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
igb5@pci0:0:20:3:       class=0x020000 card=0x1f418086 chip=0x1f418086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I354'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x2000]
    cap 10[a0] = PCI-Express 2 root endpoint max data 512(512) FLR NS
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 00900bffff78135a
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
ehci0@pci0:0:22:0:      class=0x0c0320 card=0x72708086 chip=0x1f2c8086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 USB Enhanced Host Controller'
    class      = serial bus
    subclass   = USB
    cap 01[50] = powerspec 3  supports D0 D3  current D0
    cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14
    cap 13[98] = PCI Advanced Features: FLR TP
ahci0@pci0:0:24:0:      class=0x010601 card=0x72708086 chip=0x1f328086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 AHCI SATA3 Controller'
    class      = mass storage
    subclass   = SATA
    cap 05[80] = MSI supports 1 message enabled with 1 message
    cap 01[70] = powerspec 3  supports D0 D3  current D0
    cap 12[a8] = SATA Index-Data Pair
isab0@pci0:0:31:0:      class=0x060100 card=0x72708086 chip=0x1f388086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCU'
    class      = bridge
    subclass   = PCI-ISA
    cap 09[e0] = vendor (length 12) Intel cap 1 version 0
                 features: 4 PCI-e x1 slots
none3@pci0:0:31:3:      class=0x0c0500 card=0x72708086 chip=0x1f3c8086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Atom processor C2000 PCU SMBus'
    class      = serial bus
    subclass   = SMBus
igb0@pci0:1:0:0:        class=0x020000 card=0x0000ffff chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 00900bffff78135e
    ecap 0017[1a0] = TPH Requester 1
igb1@pci0:2:0:0:        class=0x020000 card=0x0000ffff chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 00900bffff78135f
    ecap 0017[1a0] = TPH Requester 1

-------------------------------------------------------------------------------

It may be worth noting that I have observed this problem on two other systems
with igbX interfaces. pciconf for these is below.


I210 Gigabit devices
-------------------------------------------------------------------------------

igb0@pci0:7:0:0:	class=0x020000 card=0x153315d9 chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 2 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a676
    ecap 0017[1a0] = TPH Requester 1
igb1@pci0:8:0:0:	class=0x020000 card=0x153315d9 chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a677
    ecap 0017[1a0] = TPH Requester 1
pcib11@pci0:9:0:0:	class=0x060400 card=0x092115d9 chip=0x11501a03 rev=0x03 hdr=0x01
    vendor     = 'ASPEED Technology, Inc.'
    device     = 'AST1150 PCI-to-PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
    cap 05[50] = MSI supports 1 message, 64 bit 
    cap 01[78] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 10[80] = PCI-Express 1 PCI bridge max data 128(128) NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    cap 0d[a4] = PCI Bridge card=0x092115d9
    ecap 0002[100] = VC 1 max VC0
vgapci0@pci0:10:0:0:	class=0x030000 card=0x092115d9 chip=0x20001a03 rev=0x30 hdr=0x00
    vendor     = 'ASPEED Technology, Inc.'
    device     = 'ASPEED Graphics Family'
    class      = display
    subclass   = VGA
    cap 01[40] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 05[50] = MSI supports 4 messages, 64 bit 
igb2@pci0:11:0:0:	class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a678
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 8 supported
                     First VF RID Offset 0x0180, VF RID Stride 0x0004
                     VF Device ID 0x1520
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 0018[1c0] = LTR 1
    ecap 000d[1d0] = ACS 1
igb3@pci0:11:0:1:	class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a678
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 8 supported
                     First VF RID Offset 0x0180, VF RID Stride 0x0004
                     VF Device ID 0x1520
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
igb4@pci0:11:0:2:	class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a678
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 8 supported
                     First VF RID Offset 0x0180, VF RID Stride 0x0004
                     VF Device ID 0x1520
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1
igb5@pci0:11:0:3:	class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 ac1f6bffff64a678
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 8 supported
                     First VF RID Offset 0x0180, VF RID Stride 0x0004
                     VF Device ID 0x1520
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1d0] = ACS 1


I211 Gigabit devices
-------------------------------------------------------------------------------

igb0@pci0:1:0:0:	class=0x020000 card=0x00008086 chip=0x15398086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 406231ffff06a5c2
    ecap 0017[1a0] = TPH Requester 1
igb1@pci0:2:0:0:	class=0x020000 card=0x00008086 chip=0x15398086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 2 corrected
    ecap 0003[140] = Serial 1 406231ffff06a5c3
    ecap 0017[1a0] = TPH Requester 1
igb2@pci0:3:0:0:	class=0x020000 card=0x00008086 chip=0x15398086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 2 corrected
    ecap 0003[140] = Serial 1 406231ffff06a5c4
    ecap 0017[1a0] = TPH Requester 1
igb3@pci0:4:0:0:	class=0x020000 card=0x00008086 chip=0x15398086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 2 corrected
    ecap 0003[140] = Serial 1 406231ffff06a5c5
    ecap 0017[1a0] = TPH Requester 1
Comment 1 ncrogers 2019-03-25 19:04:37 UTC
Is there anything else I can do to make squashing this bug easier? I seem to recall a way to enable debug mode on an interface but I can't seem to figure this out post-iflib.
Comment 2 ncrogers 2019-03-26 16:52:43 UTC
FWIW this is also broken in latest 13-CURRENT snapshot.

FreeBSD 13.0-CURRENT r345355 GENERIC
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2019-03-30 03:21:45 UTC
Set version to earliest version issue was observed in.
Comment 4 ncrogers 2019-06-20 20:45:24 UTC
Curious if anyone else on the CC list is having the same problem or not?
Comment 5 karl 2019-06-20 21:02:56 UTC
(In reply to ncrogers from comment #4)

Maybe.

I had a very odd thing happen the other day; my PCEngines gateway/firewall machine, which has two of these interfaces in it, "disappeared" off the net.

I wasn't where the box was, so I couldn't physically check it from the console.  I was forced to have an untrained person reset it, and it came right back up.

BUT -- if the interface flapped and the upper levels got it wrong, well..... guess what -- no packets for you, which is exactly what it looked like.
Comment 6 Jeff 2019-06-20 21:03:23 UTC
(In reply to ncrogers from comment #4)
Yes, seing problem in FreeBSD 12.0-RELEASE-p5 (amd64) w/ I210 interface
Comment 7 karl 2019-06-20 21:04:36 UTC
BTW I've just updated the box (source build) to a pretty-current rev so we'll see if the problem is gone.....