Bug 265714 - igc(4) drops link under high traffic
Summary: igc(4) drops link under high traffic
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.1-STABLE
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-net (Nobody)
URL:
Keywords: IntelNetworking
Depends on:
Blocks:
 
Reported: 2022-08-08 12:40 UTC by mike
Modified: 2026-09-04 08:01 UTC (History)
12 users (show)

See Also:


Attachments
diff of output of sysctl dev.igc before and after link drop (11.56 KB, text/plain)
2022-08-08 12:40 UTC, mike
no flags Details
Patch to display NVM information similar to what's already done for igb (10.05 KB, patch)
2023-08-16 05:29 UTC, John Wehle
no flags Details | Diff
hardware info and sysctl output post test (50.99 KB, text/plain)
2024-11-25 20:51 UTC, mike
no flags Details
hw.igc.eee_setting=0 (19.05 KB, text/plain)
2024-11-26 00:19 UTC, mike
no flags Details
cederom igcdump 1786100514 (5.73 KB, application/x-xz)
2026-08-07 11:04 UTC, Tomasz "CeDeROM" CEDRO
no flags Details
igc dump while and after network stall (6.31 KB, application/x-xz)
2026-08-08 17:05 UTC, Tomasz "CeDeROM" CEDRO
no flags Details
igc stall 20260810-0140 (9.84 KB, application/x-xz)
2026-08-09 23:45 UTC, Tomasz "CeDeROM" CEDRO
no flags Details
two igc stall dumps from 20260811. (7.99 KB, application/x-xz)
2026-08-11 20:28 UTC, Tomasz "CeDeROM" CEDRO
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description mike 2022-08-08 12:40:20 UTC
Created attachment 235773 [details]
diff of output of sysctl dev.igc before and after link drop

Using a combo of netperf3 on 2 back to back RELENG_13 machines, the link will drop both at 1G and 2.5G link speeds.  Trying without FC does seem to reduce the instances of the drop, but it still happens.
Running a simple shell script to do

TARGET=10.1.255.168
I=/usr/local/bin/iperf3
REPDIR=/var/tmp/reports

while true 
do
d=`date "+%s"`
$I -t 60 -c $TARGET
sleep 2
/sbin/sysctl -a dev.igc.1 > $REPDIR/${d}a
$I -P4 -t 60 -c $TARGET
/sbin/sysctl -a dev.igc.1 > $REPDIR/${d}b
sleep 2
$I -R -t 60 -c $TARGET
/sbin/sysctl -a dev.igc.1 > $REPDIR/${d}c
sleep 2
$I  -R -P4 -t 60 -c $TARGET
/sbin/sysctl -a dev.igc.1 > $REPDIR/${d}d
done

will result in random drops of the link

igc1: link state changed to DOWN
igc1: link state changed to UP
igc1: link state changed to DOWN
igc1: link state changed to UP
igc1: link state changed to DOWN
igc1: link state changed to UP


its not clear if its the receiver or sender thats dropping it as the 2 machines are on xover cable. Discussion at 
https://lists.freebsd.org/archives/freebsd-stable/2022-August/000835.html
Comment 1 mike 2022-09-14 15:08:15 UTC
Note, running the same test with a different set of boxes that use the i226 version of the PHY works just fine. After ~ 18hrs of testing, I was not able to get the transmitting NIC to drop carrier under load




igc0@pci0:2:0:0:        class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 32, base rx80200000, size 1048576, enabled
    bar   [1c] = type Memory, range 32, base rx80300000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
                 max read 512
                 link x1(x1) speed 5.0(5.0) ASPM disabled(L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 7c2be1ffff131e81
    ecap 0018[1c0] = LTR 1
    ecap 001f[1f0] = Precision Time Measurement 1
    ecap 001e[1e0] = L1 PM Substates 1


---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-STABLE #0 stable/13-d4d8ce30d: Tue Sep 13 12:20:19 EDT 2022
    mdtancsa@topton2.sentex.ca:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c)
VT(vga): resolution 640x480
CPU: Intel(R) Celeron(R) N5105 @ 2.00GHz (1996.80-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906c0  Family=0x6  Model=0x9c  Stepping=0
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x4ff8ebbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,RDRAND>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x101<LAHF,Prefetch>
  Structured Extended Features=0x2394a2c3<FSGSBASE,TSCADJ,FDPEXC,SMEP,ERMS,NFPUSG,PQE,RDSEED,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x18400124<UMIP,WAITPKG,GFNI,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc000400<MD_CLEAR,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8019083264 (7647 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
WARNING: L1 data cache covers fewer APIC IDs than a core (0 < 1)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-119
Launching APs: 2 3 1
random: entropy device external interface
kbd0 at kbdmux0
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
smbios0: <System Management BIOS> at iomem 0x74c7b000-0x74c7b01e
smbios0: Version: 3.3, BCD Revision: 3.3
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
acpi0: <ALASKA A M I >
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 19200000 Hz quality 950
Event timer "HPET" frequency 19200000 Hz quality 550
Event timer "HPET1" frequency 19200000 Hz quality 440
Event timer "HPET2" frequency 19200000 Hz quality 440
Event timer "HPET3" frequency 19200000 Hz quality 440
Event timer "HPET4" frequency 19200000 Hz quality 440
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0x3000-0x303f mem 0x6000000000-0x6000ffffff,0x4000000000-0x400fffffff at device 2.0 on pci0
vgapci0: Boot video device
xhci0: <XHCI (generic) USB 3.0 controller> mem 0x6001100000-0x600110ffff at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <memory, RAM> at device 20.2 (no driver attached)
sdhci_pci0: <Generic SD HCI> mem 0x600111a000-0x600111afff at device 20.5 on pci0
sdhci_pci0: 1 slot(s) allocated
pci0: <serial bus> at device 21.0 (no driver attached)
pci0: <serial bus> at device 21.2 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
ahci0: <AHCI SATA controller> port 0x3090-0x3097,0x3080-0x3083,0x3060-0x307f mem 0x80500000-0x80501fff,0x80503000-0x805030ff,0x80502000-0x805027ff at device 23.0 on pci0
ahci0: AHCI v1.31 with 1 6Gbps ports, Port Multiplier not supported
ahcich1: <AHCI channel> at channel 1 on ahci0
pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
nvme0: <Generic NVMe Device> mem 0x80400000-0x80403fff at device 0.0 on pci1
pcib2: <ACPI PCI-PCI bridge> at device 28.4 on pci0
pci2: <ACPI PCI bus> on pcib2
igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80200000-0x802fffff,0x80300000-0x80303fff at device 0.0 on pci2
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: 7c:2b:e1:13:1e:81
igc0: netmap queues/slots: TX 4/1024, RX 4/1024
pcib3: <ACPI PCI-PCI bridge> at device 28.5 on pci0
pci3: <ACPI PCI bus> on pcib3
igc1: <Intel(R) Ethernet Controller I226-V> mem 0x80000000-0x800fffff,0x80100000-0x80103fff at device 0.0 on pci3
igc1: Using 1024 TX descriptors and 1024 RX descriptors
igc1: Using 4 RX queues 4 TX queues
igc1: Using MSI-X interrupts with 5 vectors
igc1: Ethernet address: 7c:2b:e1:13:1e:82
igc1: netmap queues/slots: TX 4/1024, RX 4/1024
pcib4: <ACPI PCI-PCI bridge> at device 28.6 on pci0
pci4: <ACPI PCI bus> on pcib4
igc2: <Intel(R) Ethernet Controller I226-V> mem 0x7fe00000-0x7fefffff,0x7ff00000-0x7ff03fff at device 0.0 on pci4
igc2: Using 1024 TX descriptors and 1024 RX descriptors
igc2: Using 4 RX queues 4 TX queues
igc2: Using MSI-X interrupts with 5 vectors
igc2: Ethernet address: 7c:2b:e1:13:1e:83
igc2: netmap queues/slots: TX 4/1024, RX 4/1024
pcib5: <ACPI PCI-PCI bridge> at device 28.7 on pci0
pci5: <ACPI PCI bus> on pcib5
igc3: <Intel(R) Ethernet Controller I226-V> mem 0x7fc00000-0x7fcfffff,0x7fd00000-0x7fd03fff at device 0.0 on pci5
igc3: Using 1024 TX descriptors and 1024 RX descriptors
igc3: Using 4 RX queues 4 TX queues
igc3: Using MSI-X interrupts with 5 vectors
igc3: Ethernet address: 7c:2b:e1:13:1e:84
igc3: netmap queues/slots: TX 4/1024, RX 4/1024
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
hdac0: <Intel Jasper Lake HDA Controller> mem 0x6001110000-0x6001113fff,0x6001000000-0x60010fffff at device 31.3 on pci0
pci0: <serial bus> at device 31.5 (no driver attached)

dev.igc.0.%parent: pci2
dev.igc.0.%pnpinfo: vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:2:0:0 handle=\_SB_.PC00.RP05.PXSX
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) Ethernet Controller I226-V
dev.igc.%parent:
Comment 2 John Wehle 2023-08-15 03:57:10 UTC
I recently upgrade a FreeBSD 13.2 stable system to newer hardware which included an i225 v3 NIC (it's an IOCrest M.2 card).  The system is connected to a 1Gbps switch ... every now and then the system loses link, most recent occasion was while running an installworld over NFS.  Rebooting gets things working again.
It's noted by the Intel response at:

  https://community.intel.com/t5/Ethernet-Products/Intel-Ethernet-Controller-3-I225-V-Connection-Drop/td-p/1482427

that the i225 is subject to a connection drop issue that's fixed by NVM 1.93 along with driver changes.

While currently some of the other Intel NIC drivers for FreeBSD will display EEPROM / NVM information ... i.e.:

  igb0: EEPROM V1.115-0 eTrack 0x87850000

It doesn't appear that the igc driver provides that information.  It also might be useful for the driver to display a warning if the i225 NIC is using a NVM prior to 1.93.
Comment 3 John Wehle 2023-08-16 05:29:59 UTC
Created attachment 244136 [details]
Patch to display NVM information similar to what's already done for igb

Attached is a patch to display the NVM version.  I simply took the code from the FreeBSD igb driver and transplanted it into the igc driver.  I sanity checked it against the code in the DPDK igc driver as well as smoke testing it on my system (my adapter apparently has NVM version 1.79).
Comment 4 Kevin Bowling freebsd_committer freebsd_triage 2024-11-25 01:16:36 UTC
Mike, are you able to test with say 14.2RC or newer?

But I suspect this is not a driver issue and could be a hopefully later fixed fragility in the PHY FW - either it is struggling with the cabling at the high speed, or something like EEE is interfering.
Comment 5 mike 2024-11-25 13:43:00 UTC
(In reply to Kevin Bowling from comment #4)

I dont think I have the old hardware anymore with the i225 chipset. I will however test with i226 to see how it performs
Comment 6 mike 2024-11-25 20:51:46 UTC
Created attachment 255456 [details]
hardware info and sysctl output post test

Hardware info and sysctl output
Comment 7 Kevin Bowling freebsd_committer freebsd_triage 2024-11-25 21:13:20 UTC
(In reply to mike from comment #6)
If this is on a datacenter cable plant do you have access to a cable certifier or can you eliminate that from the situation by plugging it into say a 10ft cable?

What is the switch?

Can you try with hw.igc.enable_aim=0 in loader.conf?
Comment 8 mike 2024-11-25 22:28:57 UTC
(In reply to Kevin Bowling from comment #7)
Cable is a store bought cat 6, so machine crimped.  Right now both boxes were cross over to each other. No switch. With AIM=0 I get slower performance just starting it up now
Comment 9 Kevin Bowling freebsd_committer freebsd_triage 2024-11-25 23:33:27 UTC
(In reply to mike from comment #8)
Whoops sorry AIM has been on my mind most recently.  I meant:
hw.igc.eee_setting=0
Comment 10 mike 2024-11-26 00:19:40 UTC
Created attachment 255461 [details]
hw.igc.eee_setting=0

Interesting, setting that to zero makes iperf3 fail after a few seconds with the receiver dropping / bouncing its link
Comment 11 mike 2024-11-26 00:28:27 UTC
(In reply to mike from comment #10)

this is only on the J1900 CPU.(Baytrail chipset)

It does not seem to have an impact on the Jasper Lake box
Comment 12 Kevin Bowling freebsd_committer freebsd_triage 2024-11-26 05:14:46 UTC
(In reply to mike from comment #11)
To confirm you are setting that on both sides on boot?

I am not sure what to make of the instability so far, it is sounding like the PHY FW you have is buggy which seemed to be a common problem with these chipsets early on.  There is a report of one version struggling with and then without EEE in a slightly newer version which both are eventually fixed in the later PHY FW.  The system I have access to as well as what I've heard from Netgate seem to be very stable but are running up to date NVMs which doesn't seem to be ubiquitous on consumer and builder parts.  It may be a hard battle to get ahold of the right person, but check with the manufacturer to see if they can provide an update.
Comment 13 John Wehle 2024-11-26 06:26:05 UTC
Is there any information regarding what i225 and i226 NVM versions are considered stable with the FreeBSD driver (i.e. what NVM version(s) are you and Netgate successfully using)?
Comment 14 Kevin Bowling freebsd_committer freebsd_triage 2024-11-26 12:49:04 UTC
(In reply to john from comment #13)
For I226 you will want 2.25 or 2.27 (nothing important in this one but it is the latest).  The version Mike is on, 2.17, is problematic.

I don't know the exact details on I225 but 1.79 is stable for me.
Comment 15 Kevin Bowling freebsd_committer freebsd_triage 2024-11-27 00:26:09 UTC
It's a bit of a chore but there is some unofficial ways to upgrade these https://arfoll.github.io/posts/intel_nic_nvm_update/.  I would recommend using the `-b` backup option first in case anything degrades.
Comment 16 mike 2024-11-27 14:05:40 UTC
(In reply to Kevin Bowling from comment #15)

Hi Kevin, thanks for that link. When I downloaded the latest big zip file (Release_29.4.zip) from Intel, it only seems to have I225. On the webpage
https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html
 it links to 
https://downloadmirror.intel.com/838947/Release_29.4.zip

And mentions the i226 drivers. However, I only see I225 in the zip file. Are they known by something else ? And if I dont care about the boot code, can I just update the NVM ?

% unzip -v Release_29.4.zip | grep NVMUpdatePackage
       0  Stored        0   0%  11-05-24 15:23  00000000  NVMUpdatePackage/
       0  Stored        0   0%  11-05-24 15:22  00000000  NVMUpdatePackage/700_Series/
73755774  Stored  73755774   0%  08-14-24 10:20  00000000  NVMUpdatePackage/700_Series/700Series_NVMUpdatePackage_v9_52_EFI.zip
75819910  Stored  75819910   0%  08-14-24 10:20  00000000  NVMUpdatePackage/700_Series/700Series_NVMUpdatePackage_v9_52_ESX.tar.gz
74181738  Stored  74181738   0%  08-14-24 10:20  00000000  NVMUpdatePackage/700_Series/700Series_NVMUpdatePackage_v9_52_FreeBSD.tar.gz
74142821  Stored  74142821   0%  08-14-24 10:20  00000000  NVMUpdatePackage/700_Series/700Series_NVMUpdatePackage_v9_52_Linux.tar.gz
73837560  Stored  73837560   0%  08-14-24 10:20  00000000  NVMUpdatePackage/700_Series/700Series_NVMUpdatePackage_v9_52_Windows.zip
       0  Stored        0   0%  11-05-24 15:23  00000000  NVMUpdatePackage/E810/
41827154  Stored  41827154   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackage_v4_60_EFI.zip
43895326  Stored  43895326   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackage_v4_60_ESX.tar.gz
42264187  Stored  42264187   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackage_v4_60_FreeBSD.tar.gz
42221779  Stored  42221779   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackage_v4_60_Linux.tar.gz
42279462  Stored  42279462   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackage_v4_60_Windows.zip
 9698157  Stored  9698157   0%  08-14-24 09:44  00000000  NVMUpdatePackage/E810/E810_NVMUpdatePackageAArch64_v4_60_Linux.tar.gz
       0  Stored        0   0%  11-05-24 15:23  00000000  NVMUpdatePackage/I210/
 2349985  Stored  2349985   0%  12-19-23 22:28  00000000  NVMUpdatePackage/I210/I210_NVMUpdatePackage_v2_00_EFI.zip
 4783828  Stored  4783828   0%  12-19-23 22:28  00000000  NVMUpdatePackage/I210/I210_NVMUpdatePackage_v2_00_ESX.tar.gz
 3017447  Stored  3017447   0%  12-19-23 22:28  00000000  NVMUpdatePackage/I210/I210_NVMUpdatePackage_v2_00_FreeBSD.tar.gz
 2955915  Stored  2955915   0%  12-19-23 22:28  00000000  NVMUpdatePackage/I210/I210_NVMUpdatePackage_v2_00_Linux.tar.gz
 3343227  Stored  3343227   0%  12-19-23 22:28  00000000  NVMUpdatePackage/I210/I210_NVMUpdatePackage_v2_00_Windows.zip
       0  Stored        0   0%  11-05-24 15:23  00000000  NVMUpdatePackage/I225/
 1921910  Stored  1921910   0%  03-22-24 07:30  00000000  NVMUpdatePackage/I225/I225_NVMUpdatePackage_v1_00_EFI.zip
 2534860  Stored  2534860   0%  03-22-24 07:30  00000000  NVMUpdatePackage/I225/I225_NVMUpdatePackage_v1_00_Linux.tar.gz
 2922939  Stored  2922939   0%  03-22-24 07:30  00000000  NVMUpdatePackage/I225/I225_NVMUpdatePackage_v1_00_Windows.zip
       0  Stored        0   0%  11-05-24 15:23  00000000  NVMUpdatePackage/X550/
 9860674  Stored  9860674   0%  12-19-23 22:20  00000000  NVMUpdatePackage/X550/X550_NVMUpdatePackage_v3_60_EFI.zip
12279008  Stored  12279008   0%  12-19-23 22:19  00000000  NVMUpdatePackage/X550/X550_NVMUpdatePackage_v3_60_ESX.tar.gz
10515288  Stored  10515288   0%  12-19-23 22:19  00000000  NVMUpdatePackage/X550/X550_NVMUpdatePackage_v3_60_FreeBSD.tar.gz
10453946  Stored  10453946   0%  12-19-23 22:19  00000000  NVMUpdatePackage/X550/X550_NVMUpdatePackage_v3_60_Linux.tar.gz
10825489  Stored  10825489   0%  12-19-23 22:19  00000000  NVMUpdatePackage/X550/X550_NVMUpdatePackage_v3_60_Windows.zip
Comment 17 Kevin Bowling freebsd_committer freebsd_triage 2024-11-27 20:22:09 UTC
(In reply to mike from comment #16)
No need to touch the boot code.  You can try running any of them, it will tell you whether it supports it or not.
Comment 18 Phil Dennis-Jordan 2025-02-14 14:10:48 UTC
(In reply to Kevin Bowling from comment #17)

I appear to be running into this issue with the Intel 226-V built into my Intel NUC 13 Pro. I've updated the computer's firmware to the latest version ("ANRPL357.0036").

I've also run the EFI version of the nvmupdate64e tool (V1.41.3.1) from the I225 directory of the "Intel Ethernet Series 600 NVM Update Package 30.0". This tool did detect my NIC, correctly identified as "Intel(R) Ethernet Controller I226-V" but listed the "Ver.(hex)" column as "N/A(N/A)" and "Status" as "Update not available." There isn't an I226 directory in the update package unfortunately.

I've not been able to determine a specific NIC firmware version for this device. The sysctl says `dev.igc.0.nvm: -1`, though I'm not sure if the 14.1-RELEASE-p5 kernel is able to read it out for this model?

More sysctl info:

dev.igc.0.%pnpinfo: vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x3037 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:86:0:0 handle=\_SB_.PC00.RP09.PXSX
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) Ethernet Controller I226-V


The complete dropout of handling packets persists despite the system firmware update.
Comment 19 mike 2025-02-14 14:15:38 UTC
(In reply to Phil Dennis-Jordan from comment #18)
Are you able to update to FreeBSD 14.2p1 ?  You will see a bit more info in there. 

igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80a00000-0x80afffff,0x80b00000-0x80b03fff at device 0.0 on pci1
igc0: EEPROM V2.14-0 eTrack 0x80000290
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: 60:be:b4:xx:xx:xx
igc0: netmap queues/slots: TX 4/1024, RX 4/1024

I have had pretty good stability with 14.2
Comment 20 Phil Dennis-Jordan 2025-02-17 11:11:32 UTC
I upgraded to 14.2p1 a couple days ago and dealt with the dreaded drm-kmod pkg problem by building the Intel graphics driver from ports.

I've been hitting the NIC pretty hard with sustained loads since then which would previously have tripped it, and it's going strong so far. Relevant dmesg boot output with the new version:

igc0: <Intel(R) Ethernet Controller I226-V> mem 0x84200000-0x842fffff,0x84300000-0x84303fff at device 0.0 on pci2
igc0: EEPROM V2.17-0 eTrack 0x80000308
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: 48:21:0b:59:36:2d
igc0: netmap queues/slots: TX 4/1024, RX 4/1024

Looking through the commit logs for igc(4), it seems interrupt management has changed between 14.1 and 14.2, with the introduction of "AIM". Does this AIM mode effectively work around a bug in the NIC's firmware or is there a bug the non-AIM interrupt management in the driver that caused queues not to be processed in specific edge cases? (I didn't spot any other commits that might be relevant, but I'm new to this code; I also notice AIM remains disabled for speeds slower than gigabit, so could the issue still occur there?)
Comment 21 Kevin Bowling freebsd_committer freebsd_triage 2025-02-18 06:26:41 UTC
(In reply to Phil Dennis-Jordan from comment #20)
I am not aware of any changes that would impact this, but we still don't know what the problem is.  The largest known I226 user (netgate) has no issues.

The only thing I can say for certain so far is you all should be bugging the HW vendors for NVM 2.25 or 2.27 for I226 otherwise there are known issues around both EEE enabled and disabled.  It would be nice to know what happens once that is achieved, to level if we are chasing a FreeBSD bug and not fixed microcode issues.
Comment 22 Loïc Bartoletti freebsd_committer freebsd_triage 2025-03-03 15:46:28 UTC
I can confirm that igc hangs on my ASUS ROG STRIX X670E-A GAMING WIFI motherboard.

I don't know why, sometimes with big downloads (pkg install, download, etc) sometimes without raeson...

I have patched bios/motherboard/etc to latest versions...

However, as mentioned in a comment, disabling aspm in the bios is a solution ; at least since 2 days it's not hanging.
Comment 23 mike 2025-03-03 15:51:54 UTC
(In reply to Loïc Bartoletti from comment #22)
what does dmesg show for the version of the igc nic ? 
e.g. on mine, I have 
igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80200000-0x802fffff,0x80300000-0x80303fff at device 0.0 on pci2
igc0: EEPROM V2.14-0 eTrack 0x80000290
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: 60:be:b4:16:13:f8
igc0: netmap queues/slots: TX 4/1024, RX 4/1024
Comment 24 Loïc Bartoletti freebsd_committer freebsd_triage 2025-03-04 06:23:14 UTC
(In reply to mike from comment #23)

igc0: <Intel(R) Ethernet Controller I225-V> mem 0xf5200000-0xf52fffff,0xf5300000-0xf5303fff at device 0.0 on pci10
igc0: EEPROM V1.82-0 eTrack 0x80000266
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: c8:7f:54:68:ac:2c
igc0: netmap queues/slots: TX 4/1024, RX 4/1024
igc0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igc0 (ifp 0xfffff80001cdb000), ignoring.
igc0: link state changed to DOWN
igc0: link state changed to UP
Comment 25 Tinkerer 2025-03-31 23:57:09 UTC
Hey Folks - Just wanted to cast my stone into the pond regarding this issue. I'm running OPNsense on FreeBSD 14.2. I have an HP Flex IO i225 NIC installed in my HP Elitedesk 800 G6 Mini. Current firmware version is 1.57. I am unable to upgrade the nvm via nvmupdatemanager64 since the firmware is HP proprietary, with 1.57 being the highest available version from HP.

I run the NIC at 2.5 Gbe. Under load, my speeds will drop down to around 500-600 Mb/s with large amounts of retries in iperf3. I can induce the speed drop by rapidly starting/stopping iperf transfers.

I've tried everything - tunables, BIOS power settings/ASPM, AIM settings, EEE settings. Nothing has remedied the issue.

I have an identical PC with the same NIC installed, which runs Linux (Proxmox). The NIC in this machine does not experience any drops. This leads me to believe that the Linux drivers are able to compensate for any NIC hardware/microcode issues. The firmware of the NIC in the Linux box is also 1.57.

My pciconf output:

class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x103c subdevice=0x87b9

Hopefully as time goes on there may be a takeaway from the Linux driver that FreeBSD implements. My feeling is that most folks experiencing issues with this NIC have variants that are not OEM Intel.
Comment 26 Kevin Bowling freebsd_committer freebsd_triage 2025-04-01 07:04:19 UTC
(In reply to Tinkerer from comment #25)
That sounds like a hashing collision where multiple flows land on a single core/queue and there is no back pressure.  Not related to this bug at all which is link completely unavailable.
Comment 27 Tinkerer 2025-04-03 01:27:12 UTC
(In reply to Kevin Bowling from comment #26)
I apologize - I didn't mean to muddle the thread with something off-base, but thought it may be something related. Please disregard
Comment 28 Kevin Bowling freebsd_committer freebsd_triage 2025-04-03 06:49:38 UTC
(In reply to Tinkerer from comment #27)
The TCP RACK stack has a mitigation for that https://man.freebsd.org/cgi/man.cgi?query=tcp_rack&manpath=FreeBSD+14.2-RELEASE+and+Ports but yes it is not helpful in this PR.
Comment 29 Phil Dennis-Jordan 2025-04-18 14:30:02 UTC
I've just encountered this issue again after it's been quiet for a long time - I'm using the device in a different location where the negotiated link speed is only 1Gbps versus the usual 2.5Gbps, so perhaps it's easier to still hit the problem at lower media speeds.

I've updated to the very latest NUC 13 platform firmware (ANRPL357.0037) but this still seems to contain the problematic I226V EEPROM version 2.17. I've submitted a support request to ASUS, here's hoping they provide an update path.
Comment 30 Kevin Bowling freebsd_committer freebsd_triage 2025-04-18 17:24:46 UTC
(In reply to Phil Dennis-Jordan from comment #29)
It would be _really_ helpful to get a report, positive or negative, of what happens once a system with this problem is brought up to known good NVM.

I did have some private discussions with Intel and they agree it is possible this is NVM related and had no specific insight into other possibilities.
Comment 31 Konstantin Belousov freebsd_committer freebsd_triage 2025-04-19 16:26:04 UTC
It seems I was also just beaten by the link drop on 226-V:
igc1: <Intel(R) Ethernet Controller I226-V> mem 0xdca00000-0xdcafffff,0xdcb00000
-0xdcb03fff at device 0.0 on pci3
igc1: attach_pre capping queues at 4
igc1: EEPROM V2.23-0 eTrack 0x80000397
igc1: Using 1024 TX descriptors and 1024 RX descriptors
igc1: msix_init qsets capped at 4
igc1: intr CPUs: 8 queue msgs: 4 admincnt: 1
igc1: Using 4 RX queues 4 TX queues
igc1: attempting to allocate 5 MSI-X vectors (5 supported)

Running stable/14 as of week old, link is 1Gb.  Some combination of
ifconfig down/up and physical cable re-insertion fixed the problem for now.

Is it 'recommended' to get the link to 2.5G? I can insert the dumb switch between
the machine and the managed port.
Comment 32 karl 2025-08-17 23:50:00 UTC
I've got a little N-150 "minimal PC" box here with dual I226-v ports -- it comes up with 2.17 for the EEPROM.

igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80c00000-0x80cfffff,0x80d00000-0x80d03fff at device 0.0 on pci1
igc0: EEPROM V2.17-0 eTrack 0x80000303
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: c8:ff:bf:05:95:e2
igc0: netmap queues/slots: TX 4/1024, RX 4/1024

....
$ ifconfig igc0
igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4e427bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether c8:ff:bf:05:95:e2
        inet 192.168.10.50 netmask 0xffffff00 broadcast 192.168.10.255
        media: Ethernet autoselect (2500Base-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

If I turn off TSO and LRO (intended thing to do for the outside interface on a gateway/firewall) the only change is a slight increase in CPU use, as expected.

I've had zero trouble under 14.3-STABLE (built about a week ago) and I've hammered it pretty hard with iperf3 on 2.5Gbps connections.... zero hangs, drops or other misbehavior.

The "other end" I'm testing against is a server on a 10GbE link through a Mellanox card (mce driver) into a switch with both 10G and 2.5G ports.

(This is in contrast to a similar machine from a different vendor I have with dual RealTek 2.5G ports that is unusably bad; random resets occur on the kernel-included driver under load and with the kmod driver some versions if I dig through the older package sets sort of work but the "best" ones seem to be "stable" but produce insane retry counts on iperf and very random-latency performance on real workloads.)

I am not setting anything special (e.g. hw.pci.enable_aspm=0) or similar.
Comment 33 John Wehle 2025-09-25 03:35:29 UTC
It looks like instructions and pointers to the necessary tools / firmware images for upgrading the NVM is available at:

https://forum.opnsense.org/index.php?topic=48695.msg246912#msg246912
Comment 34 Tomasz "CeDeROM" CEDRO 2026-02-17 22:51:32 UTC
Hello world, I just started searching bugzilla for igc0 related issues, looks like I have this problem too on 14.3-RELEASE-p8 with igc0 onboard ASUS TUF GAMING Z890-PRO WIFI (latest bios 2401 09/12/2025) with Intel Core Ultra9 285K. Quite frequently ethernet just stops working, no visible link down or anything like this. Simple ifconfig igc0 down/up helps. But this gets annoying :-P

I tried playing with various sysctl settings, no difference.

Looks like it uses already latest EEPROM V2.23-0 eTrack 0x8000039d.

The problem may be even more intensive as I plan soon to switch to 8G ISP link and will use 2.5G instead of 1G ethernet. Maybe not.

Any hints welcome, details below, good to see I am not alone :-P


ifconfig:
igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e427bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	inet XXX netmask 0xXXX broadcast XXX
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

sysctl:
[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0xb8000000-0xb80fffff,0xb8100000-0xb8103fff at device 0.0 on pci7
[1] igc0: EEPROM V2.23-0 eTrack 0x8000039d
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
<6>[1] igc0: Ethernet address: XXX
<6>[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024
<6>[28] igc0: link state changed to UP
<6>[28] igc0: link state changed to DOWN
<118>[28] Starting Network: lo0 igc0.
<118>[28] igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
<6>[32] igc0: link state changed to UP
<118>[33] DHCPDISCOVER on igc0 to 255.255.255.255 port 67 interval 6
<118>[39] DHCPDISCOVER on igc0 to 255.255.255.255 port 67 interval 10
<118>[39] DHCPREQUEST on igc0 to 255.255.255.255 port 67
<6>[13490] igc0: link state changed to DOWN
<6>[13493] igc0: link state changed to UP
device	igc
kern.wait_dequeue_sigchld: 1
irq151: igc0:rxq0:325 @cpu0(domain0): 1575337
irq152: igc0:rxq1:327 @cpu1(domain0): 364224
irq153: igc0:rxq2:329 @cpu2(domain0): 950965
irq154: igc0:rxq3:331 @cpu3(domain0): 949877
irq155: igc0:aq:333 @cpu0(domain0): 4
hw.igc.max_interrupt_rate: 4000
hw.igc.enable_aim: 1
hw.igc.eee_setting: 1
hw.igc.sbp: 1
hw.igc.smart_pwr_down: 0
hw.igc.disable_crc_stripping: 0
dev.igc.0.wake: 0
dev.igc.0.interrupts.rx_desc_min_thresh: 0
dev.igc.0.interrupts.asserts: 3840741
dev.igc.0.mac_stats.tso_txd: 4745
dev.igc.0.mac_stats.tx_frames_1024_1522: 1958563
dev.igc.0.mac_stats.tx_frames_512_1023: 199916
dev.igc.0.mac_stats.tx_frames_256_511: 397764
dev.igc.0.mac_stats.tx_frames_128_255: 224216
dev.igc.0.mac_stats.tx_frames_65_127: 1568506
dev.igc.0.mac_stats.tx_frames_64: 2062
dev.igc.0.mac_stats.mcast_pkts_txd: 0
dev.igc.0.mac_stats.bcast_pkts_txd: 252
dev.igc.0.mac_stats.good_pkts_txd: 4351027
dev.igc.0.mac_stats.total_pkts_txd: 4351027
dev.igc.0.mac_stats.good_octets_txd: 2847437928
dev.igc.0.mac_stats.good_octets_recvd: 9267959132
dev.igc.0.mac_stats.rx_frames_1024_1522: 5992756
dev.igc.0.mac_stats.rx_frames_512_1023: 353950
dev.igc.0.mac_stats.rx_frames_256_511: 166485
dev.igc.0.mac_stats.rx_frames_128_255: 177195
dev.igc.0.mac_stats.rx_frames_65_127: 172869
dev.igc.0.mac_stats.rx_frames_64: 1149
dev.igc.0.mac_stats.mcast_pkts_recvd: 773
dev.igc.0.mac_stats.bcast_pkts_recvd: 4296
dev.igc.0.mac_stats.good_pkts_recvd: 6864404
dev.igc.0.mac_stats.total_pkts_recvd: 6934131
dev.igc.0.mac_stats.mgmt_pkts_txd: 0
dev.igc.0.mac_stats.mgmt_pkts_drop: 0
dev.igc.0.mac_stats.mgmt_pkts_recvd: 0
dev.igc.0.mac_stats.unsupported_fc_recvd: 0
dev.igc.0.mac_stats.xoff_txd: 0
dev.igc.0.mac_stats.xoff_recvd: 0
dev.igc.0.mac_stats.xon_txd: 0
dev.igc.0.mac_stats.xon_recvd: 0
dev.igc.0.mac_stats.alignment_errs: 0
dev.igc.0.mac_stats.crc_errs: 0
dev.igc.0.mac_stats.recv_errs: 0
dev.igc.0.mac_stats.recv_jabber: 0
dev.igc.0.mac_stats.recv_oversize: 0
dev.igc.0.mac_stats.recv_fragmented: 0
dev.igc.0.mac_stats.recv_undersize: 0
dev.igc.0.mac_stats.recv_no_buff: 0
dev.igc.0.mac_stats.recv_length_errors: 0
dev.igc.0.mac_stats.missed_packets: 60501
dev.igc.0.mac_stats.defer_count: 0
dev.igc.0.mac_stats.sequence_errors: 0
dev.igc.0.mac_stats.symbol_errors: 0
dev.igc.0.mac_stats.collision_count: 0
dev.igc.0.mac_stats.late_coll: 0
dev.igc.0.mac_stats.multiple_coll: 0
dev.igc.0.mac_stats.single_coll: 0
dev.igc.0.mac_stats.excess_coll: 0
dev.igc.0.queue_rx_3.rx_irq: 0
dev.igc.0.queue_rx_3.rxd_tail: 797
dev.igc.0.queue_rx_3.rxd_head: 798
dev.igc.0.queue_rx_3.interrupt_rate: 20832
dev.igc.0.queue_rx_2.rx_irq: 0
dev.igc.0.queue_rx_2.rxd_tail: 498
dev.igc.0.queue_rx_2.rxd_head: 499
dev.igc.0.queue_rx_2.interrupt_rate: 20832
dev.igc.0.queue_rx_1.rx_irq: 0
dev.igc.0.queue_rx_1.rxd_tail: 420
dev.igc.0.queue_rx_1.rxd_head: 421
dev.igc.0.queue_rx_1.interrupt_rate: 20832
dev.igc.0.queue_rx_0.rx_irq: 0
dev.igc.0.queue_rx_0.rxd_tail: 992
dev.igc.0.queue_rx_0.rxd_head: 993
dev.igc.0.queue_rx_0.interrupt_rate: 20832
dev.igc.0.queue_tx_3.tx_irq: 0
dev.igc.0.queue_tx_3.txd_tail: 1011
dev.igc.0.queue_tx_3.txd_head: 1011
dev.igc.0.queue_tx_3.interrupt_rate: 20832
dev.igc.0.queue_tx_2.tx_irq: 0
dev.igc.0.queue_tx_2.txd_tail: 121
dev.igc.0.queue_tx_2.txd_head: 121
dev.igc.0.queue_tx_2.interrupt_rate: 20832
dev.igc.0.queue_tx_1.tx_irq: 0
dev.igc.0.queue_tx_1.txd_tail: 871
dev.igc.0.queue_tx_1.txd_head: 871
dev.igc.0.queue_tx_1.interrupt_rate: 20832
dev.igc.0.queue_tx_0.tx_irq: 0
dev.igc.0.queue_tx_0.txd_tail: 594
dev.igc.0.queue_tx_0.txd_head: 594
dev.igc.0.queue_tx_0.interrupt_rate: 20832
dev.igc.0.fc_low_water: 32752
dev.igc.0.fc_high_water: 32768
dev.igc.0.rx_control: 71335938
dev.igc.0.device_control: 1209796161
dev.igc.0.watchdog_timeouts: 0
dev.igc.0.rx_overruns: 0
dev.igc.0.link_irq: 4
dev.igc.0.dropped: 0
dev.igc.0.eee_control: 1
dev.igc.0.tso_tcp_flags_mask_last_segment: 3967
dev.igc.0.tso_tcp_flags_mask_middle_segment: 3958
dev.igc.0.tso_tcp_flags_mask_first_segment: 4086
dev.igc.0.dmac: 0
dev.igc.0.rs_dump: 0
dev.igc.0.reg_dump: General Registers
dev.igc.0.fc: 3
dev.igc.0.debug: -1
dev.igc.0.fw_version: EEPROM V2.23-0 eTrack 0x8000039d
dev.igc.0.enable_aim: 1
dev.igc.0.nvm: -1
dev.igc.0.iflib.rxq3.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq3.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq3.rxq_fl0.cidx: 798
dev.igc.0.iflib.rxq3.rxq_fl0.pidx: 797
dev.igc.0.iflib.rxq3.cpu: 3
dev.igc.0.iflib.rxq2.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq2.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq2.rxq_fl0.cidx: 499
dev.igc.0.iflib.rxq2.rxq_fl0.pidx: 498
dev.igc.0.iflib.rxq2.cpu: 2
dev.igc.0.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq1.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq1.rxq_fl0.cidx: 421
dev.igc.0.iflib.rxq1.rxq_fl0.pidx: 420
dev.igc.0.iflib.rxq1.cpu: 1
dev.igc.0.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq0.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq0.rxq_fl0.cidx: 993
dev.igc.0.iflib.rxq0.rxq_fl0.pidx: 992
dev.igc.0.iflib.rxq0.cpu: 0
dev.igc.0.iflib.txq3.r_abdications: 0
dev.igc.0.iflib.txq3.r_restarts: 0
dev.igc.0.iflib.txq3.r_stalls: 0
dev.igc.0.iflib.txq3.r_starts: 1389
dev.igc.0.iflib.txq3.r_drops: 0
dev.igc.0.iflib.txq3.r_enqueues: 1389
dev.igc.0.iflib.txq3.ring_state: pidx_head: 1242 pidx_tail: 1242 cidx: 1242 state: IDLE
dev.igc.0.iflib.txq3.txq_cleaned: 3017
dev.igc.0.iflib.txq3.txq_processed: 3057
dev.igc.0.iflib.txq3.txq_in_use: 42
dev.igc.0.iflib.txq3.txq_cidx_processed: 1009
dev.igc.0.iflib.txq3.txq_cidx: 969
dev.igc.0.iflib.txq3.txq_pidx: 1011
dev.igc.0.iflib.txq3.no_tx_dma_setup: 0
dev.igc.0.iflib.txq3.txd_encap_efbig: 0
dev.igc.0.iflib.txq3.tx_map_failed: 0
dev.igc.0.iflib.txq3.no_desc_avail: 0
dev.igc.0.iflib.txq3.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq3.m_pullups: 0
dev.igc.0.iflib.txq3.mbuf_defrag: 0
dev.igc.0.iflib.txq3.cpu: 3
dev.igc.0.iflib.txq2.r_abdications: 0
dev.igc.0.iflib.txq2.r_restarts: 0
dev.igc.0.iflib.txq2.r_stalls: 0
dev.igc.0.iflib.txq2.r_starts: 751
dev.igc.0.iflib.txq2.r_drops: 0
dev.igc.0.iflib.txq2.r_enqueues: 752
dev.igc.0.iflib.txq2.ring_state: pidx_head: 0722 pidx_tail: 0722 cidx: 0722 state: IDLE
dev.igc.0.iflib.txq2.txq_cleaned: 2127
dev.igc.0.iflib.txq2.txq_processed: 2167
dev.igc.0.iflib.txq2.txq_in_use: 42
dev.igc.0.iflib.txq2.txq_cidx_processed: 119
dev.igc.0.iflib.txq2.txq_cidx: 79
dev.igc.0.iflib.txq2.txq_pidx: 121
dev.igc.0.iflib.txq2.no_tx_dma_setup: 0
dev.igc.0.iflib.txq2.txd_encap_efbig: 0
dev.igc.0.iflib.txq2.tx_map_failed: 0
dev.igc.0.iflib.txq2.no_desc_avail: 0
dev.igc.0.iflib.txq2.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq2.m_pullups: 0
dev.igc.0.iflib.txq2.mbuf_defrag: 0
dev.igc.0.iflib.txq2.cpu: 2
dev.igc.0.iflib.txq1.r_abdications: 0
dev.igc.0.iflib.txq1.r_restarts: 0
dev.igc.0.iflib.txq1.r_stalls: 0
dev.igc.0.iflib.txq1.r_starts: 6495
dev.igc.0.iflib.txq1.r_drops: 0
dev.igc.0.iflib.txq1.r_enqueues: 6495
dev.igc.0.iflib.txq1.ring_state: pidx_head: 1772 pidx_tail: 1772 cidx: 1772 state: IDLE
dev.igc.0.iflib.txq1.txq_cleaned: 13117
dev.igc.0.iflib.txq1.txq_processed: 13157
dev.igc.0.iflib.txq1.txq_in_use: 42
dev.igc.0.iflib.txq1.txq_cidx_processed: 869
dev.igc.0.iflib.txq1.txq_cidx: 829
dev.igc.0.iflib.txq1.txq_pidx: 871
dev.igc.0.iflib.txq1.no_tx_dma_setup: 0
dev.igc.0.iflib.txq1.txd_encap_efbig: 0
dev.igc.0.iflib.txq1.tx_map_failed: 0
dev.igc.0.iflib.txq1.no_desc_avail: 0
dev.igc.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq1.m_pullups: 0
dev.igc.0.iflib.txq1.mbuf_defrag: 0
dev.igc.0.iflib.txq1.cpu: 1
dev.igc.0.iflib.txq0.r_abdications: 0
dev.igc.0.iflib.txq0.r_restarts: 0
dev.igc.0.iflib.txq0.r_stalls: 0
dev.igc.0.iflib.txq0.r_starts: 3315
dev.igc.0.iflib.txq0.r_drops: 0
dev.igc.0.iflib.txq0.r_enqueues: 3316
dev.igc.0.iflib.txq0.ring_state: pidx_head: 0212 pidx_tail: 0212 cidx: 0212 state: IDLE
dev.igc.0.iflib.txq0.txq_cleaned: 7718
dev.igc.0.iflib.txq0.txq_processed: 7758
dev.igc.0.iflib.txq0.txq_in_use: 44
dev.igc.0.iflib.txq0.txq_cidx_processed: 590
dev.igc.0.iflib.txq0.txq_cidx: 550
dev.igc.0.iflib.txq0.txq_pidx: 594
dev.igc.0.iflib.txq0.no_tx_dma_setup: 0
dev.igc.0.iflib.txq0.txd_encap_efbig: 0
dev.igc.0.iflib.txq0.tx_map_failed: 0
dev.igc.0.iflib.txq0.no_desc_avail: 0
dev.igc.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq0.m_pullups: 0
dev.igc.0.iflib.txq0.mbuf_defrag: 0
dev.igc.0.iflib.txq0.cpu: 0
dev.igc.0.iflib.override_nrxds: 0
dev.igc.0.iflib.override_ntxds: 0
dev.igc.0.iflib.allocated_msix_vectors: 5
dev.igc.0.iflib.use_extra_msix_vectors: 0
dev.igc.0.iflib.use_logical_cores: 0
dev.igc.0.iflib.separate_txrx: 0
dev.igc.0.iflib.core_offset: 0
dev.igc.0.iflib.tx_abdicate: 0
dev.igc.0.iflib.rx_budget: 0
dev.igc.0.iflib.disable_msix: 0
dev.igc.0.iflib.override_qs_enable: 0
dev.igc.0.iflib.override_nrxqs: 0
dev.igc.0.iflib.override_ntxqs: 0
dev.igc.0.iflib.driver_version: 1
dev.igc.0.%iommu: rid=0x8400
dev.igc.0.%parent: pci7
dev.igc.0.%pnpinfo: vendor=0x8086 device=0x125c subvendor=0x1043 subdevice=0x8867 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:132:0:0 handle=\_SB_.PC02.RP04.PXSX
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) Ethernet Controller I226-V
dev.igc.%parent:

dmesg:
[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0xb8000000-0xb80fffff,0xb8100000-0xb8103fff at device 0.0 on pci7
[1] igc0: EEPROM V2.23-0 eTrack 0x8000039d
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
[1] igc0: Ethernet address: XXX
[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024
Comment 35 Tomasz "CeDeROM" CEDRO 2026-02-19 15:45:31 UTC
* There was BIOS 2401 -> 3002 update for my motherboard.
* I226-V still remains EEPROM V2.23-0 eTrack 0x8000039d.
* Tried windoze drivers and tools update from Intel version 31.0 bundle [1], tool works, but they provide only firmware update for i225 cards.
* Intel for some reason does not share anything new for i226-V to the public although known issues many complains and requests they hide possible updates/fixes behind tons of bureaucracy which actually actively blocks the fix [2].
* I am in contact with ASUS maybe they will be able to provide some public fixes for the i226-V card.

[1] https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html
[2] https://community.intel.com/t5/Ethernet-Products/Where-can-i-download-I226-V-NVM-and-Flash-update-tool-and-image/m-p/1737553/emcs_t/S2h8ZW1haWx8dG9waWNfc3Vic2NyaXB0aW9ufE1MUkM5UURJSUQ5U0pSfDE3Mzc1NTN8U1VCU0NSSVBUSU9OU3xoSw
Comment 36 Tomasz "CeDeROM" CEDRO 2026-02-20 20:18:37 UTC
Okay, so the problem still exists on my mobo bios 3002:
* It is rare on slow ISP (~50M).
* It is frequent on fast ISP (1G).
* It may be related with vm (vbox) or just more traffic is generated.
* ifconfig igc0 show all work fine, dmesg shows no problems, but there is no traffic, simple ifconfig igc0 down/up fixes the issue but this is not possible for remote connections.

I am in contact with ASUS Poland they just forwarded this case to HQ, really good support I must say, lets hope they will find the solution, maybe obtain new firmware from Intel or push them for the fix :-)
Comment 37 Kevin Bowling freebsd_committer freebsd_triage 2026-02-21 06:47:46 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #36)
Very interested to see if we can finally pin this to firmware or not.  Hopefully they can get you the latest bundle.
Comment 38 Tomasz "CeDeROM" CEDRO 2026-05-13 17:34:25 UTC
Last updates from opnsense forum [1] indicates the problem is in the firmware and best fix is to completely disable ASPM for this device and this would fix whatever firmware is on the card.

I am really angry at this moment because of these network stops that require manual network restarts. This is unacceptable when streaming, during online meetings, etc. I am considering disabling this i226-V crap and just use different network adapter.

I saw intel released 31.1 drivers update on 2026-03-20 [2]. But again I can see only firmware update for i225 not i226. Will run the windoze installer and see if that fixes anything on FreeBSD. If not then Intel goes to trash. I am very disappointed.

[1] https://forum.opnsense.org/index.php?topic=48695.210
[2] https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html
Comment 39 Tomasz "CeDeROM" CEDRO 2026-05-13 17:35:49 UTC
Forgot to mention that my ASUS mobo bios only allows whole system level ASPM control, bot single device (i.e. this NIC). This impacts other system components.
Comment 40 Tomasz "CeDeROM" CEDRO 2026-05-13 17:39:26 UTC
2.32 firmeware is here https://github.com/BillyCurtis/Intel-i226-V-NVM-Firmware?tab=readme-ov-file#readme
Comment 41 Kevin Bowling freebsd_committer freebsd_triage 2026-05-13 18:51:05 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #40)
Are you able to test those firmwares to see if they fix the issue?  I could see a couple routes if they help 1) increase documentation so people know where and how to update (good) 2) add some warnings and chicken bits for ASPM or whatever else to the driver if down level (ugly but functional)
Comment 42 Tomasz "CeDeROM" CEDRO 2026-05-13 18:54:37 UTC
There was a BIOS update for my mobo 3002 -> 3012 -> 3020, now working on the 3020 will check if this was not a BIOS issue, but i226-V NIC fw remains the same 2.32, so ASUS did nothing in this area.

Updating Intel drivers to 31.1 on windoze also did not update NIC fw. There is new fw update package but only for i225 cards.

Intel seems to be ashamed of this i226-V product :-(

[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0xb2000000-0xb20fffff,0xb2100000-0xb2103fff at device 0.0 on pci7
[1] igc0: EEPROM V2.23-0 eTrack 0x8000039d
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
[1] igc0: Ethernet address: XXX
[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024
Comment 43 Tomasz "CeDeROM" CEDRO 2026-05-13 19:01:11 UTC
Okay this seems to be very good solution, very nice step by step instruction, and the binary updater is for FreeBSD, will try that now, posting now just in case I will get offline for some time :D

https://github.com/nightcomdev/opnsense/tree/main/i226-firmware-upgrade

% file nvmupdate64e
nvmupdate64e: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.1, FreeBSD-style, stripped

Also posting content below just in case it disappears from gh:

Intel i226-V Firmware v2.32 upgrade pack

Link to Firmware upgrade pack for i226-V with image v2.32 (1MB version)

https://github.com/nightcomdev/opnsense/raw/refs/heads/main/i226-firmware-upgrade/i226.firmware.v2.32.zip

This pack contains:

* This README file
* Billy Curtis i226-V v2.32 1MB bin
* Example nvm.cfg file
*  nvmupdate64e util from the Intel 34.0 download bundle
* Intel doc on using nvmupdate64e

Check config file and adjust to your hardware!
Steps to upgrade your ethernet card

* Copy and unzip files to your OPNsense router
* Check what card do you have dmesg |grep igc0 change number if you have multiple cards igc1, igc2...
* Check nvm.cfg is it correct with informations from dmesg |grep igc0 if not then adjust to your hardware informations
* Copy MAC address and paste it in next line without :
* ./nvmupdate64e -b -l nvm.log -m 00E0B468DCBC -f -u -c nvm.cfg
* Run dmesg |grep igc0 and confirm that firmware changed from 2.17 -> 2.32
* Firmware is upgraded.
* Unplug power cord from device for 10-15sec and then turn device back on.

Be aware that when you upgrade your LAN NIC you will loose connection if you doing this remotely, in this case leave it for 1-2min to finish task and then REBOOT

Note

If your port is not working it's adviced to unplug power cord from device for 10-15sec and turn it back on. I did that step even after successful upgrade.

More info about i226-V firmware upgrade
Topic on OPNsense forum

https://forum.opnsense.org/index.php?topic=48695.0
For more informations about upgrading i226-V firmware or other versions of images please visit repository of BillyCurtis.

https://github.com/BillyCurtis/Intel-I226-V-NVM-Firmware
How to use Intel® Ethernet NVM Update Tool

http://tmei-net.com/technology-sharing/how-do-use-intel-ethernet-nvm-update-tool.html
Comment 44 Florian Smeets freebsd_committer freebsd_triage 2026-05-13 19:05:12 UTC
(In reply to Kevin Bowling from comment #41)
I did those firmware upgrades a couple of months ago on a system with 6 I226-V ports (3 in use), and it finally made the system stable.
Comment 45 Tomasz "CeDeROM" CEDRO 2026-05-13 20:16:08 UTC
Hello i226-V V2.32-0 eTrack 0x80000425 world, hope the problem is gone now, time will tell :-)

Big shame this solution does not come from Intel directly!!!

HOST: 

FreeBSD hexagon 14.4-RELEASE-p3 FreeBSD 14.4-RELEASE-p3 GENERIC amd64

DMESG:

[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0xb2000000-0xb20fffff,0xb2100000-0xb2103fff at device 0.0 on pci7
[1] igc0: EEPROM V2.32-0 eTrack 0x80000425
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
[1] igc0: Ethernet address: [REDACTED]
[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024


Used package and instructions from here:

https://github.com/nightcomdev/opnsense/tree/main/i226-firmware-upgrade

In addition to the original instruction just three more hints:

1. `cp nvm.cfg.txt nvm.cfg` to get the tool configuration file and keep the original.

2. Update SUBVENDOR and SUBDEVICE fields in the nvm.cfg file by running `pciconf  -lv | grep -A2 -B2 226-V` to see the correct values for your NIC. Without this information firmware update application won't be able to find the card.

3. Update nvm.cfg REPLACES field to match your current eTrack.


UPGRADE LOG:

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.43.20.0
Copyright(C) 2013 - 2025 Intel Corporation.

./nvmupdate64e -b -l nvm.log -m [REDACTED] -u -c nvm.cfg

Config file read.
Inventory
[00:132:00:00]: Intel(R) Ethernet Controller I226-V
        Alternate MAC address is not set.
        Flash inventory started.
        Shadow RAM inventory started.
        Shadow RAM inventory finished.
        Flash inventory finished.
Update
[00:132:00:00]: Intel(R) Ethernet Controller I226-V
        Creating backup images in directory: [REDACTED].
        Backup images created.
        Flash update started.
        NVM verification started.
        Shadow RAM verification started.
        Shadow RAM verification finished.
        Flash verification started.
Difference found in module Invalid at offset 0x21FF - update required.
        Flash verification finished.
        NVM update is required.
        NVM verification finished.
Error:          Flash update failed.
        Device update failed.
Update security revisions
[00:132:00:00]: Intel(R) Ethernet Controller I226-V
        Skipping update minimum security revisions.
Update VPD with VPD template
[00:132:00:00]: Intel(R) Ethernet Controller I226-V
        Skipping VPD update with VPD template.


Just in case we need the nvm.cfg content later on here it goes:

CURRENT FAMILY: 1.0.0
CONFIG VERSION: 1.20.0

; NIC device
BEGIN DEVICE
DEVICENAME: Intel(R) Ethernet Controller I226-V
VENDOR: 8086
DEVICE: 125C
SUBVENDOR: 1043
SUBDEVICE: 8867
NVM IMAGE: FXVL_125C_V_1MB_2.32.bin
EEPID: 80000425
RESET TYPE: REBOOT
REPLACES: 8000039D
END DEVICE
Comment 46 Tomasz "CeDeROM" CEDRO 2026-05-13 20:20:23 UTC
One more thing, the backup firmware is 2MB in size, the provided firmware is 1MB, let's see how it works ;-)
Comment 47 Tomasz "CeDeROM" CEDRO 2026-05-14 16:21:22 UTC
So far so good, standard workstation load included:
* downloading over 200GB content in multiple streams.
* vbox machine start, load, stop, start, load, stop, etc. <- that was usually the most often cause of igc failures.
* standard CPU loads with all cores firmware code compilation.
* burning 100GB bdxl disks with no problem for ~10h/disk.

All this with ASPM enabled in BIOS and set to L2!! :-)

I even tried to run Cyberpunk 2077 game in Wine on top of all that just to push the machine to its limits and make some PCI traffic to the GPU. The game runs smooth and this new Wine stuff is amazing! However old stuff like PLC SDK due to dedicated drivers is better when contained in the VM (runs solwer but comms drivers work).

After BIOS update computer seems to run smoother too. Yesterday was a good day :-)

2.32 firmware seems to fix the issue for my i226-V :-)

This is also good information because I will soon get 7GBps WAN, so I would be able to see how i226-V behaves at full 2.5GBps.. and the next NIC will be 10GBps.. just not sure if Intel ;-)
Comment 48 Sean Farley freebsd_committer freebsd_triage 2026-05-18 01:12:16 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #46)

From something I read, the 2MB file can either mean the backup firmware is actually 2MB or it is 1MB but padded with zeroes because the tool is written to handle 2MB regardless of the source.  In my case, I ran `split -b 1M` on the firmware and found that xaa and xab are exactly the same.  While I did not try it, my system would need only the first half of the backup file to actually be able to restore the backup.

BTW, if anyone runs into a corrupted state, there is a better, yet harder to find, tool called eeupdate64e (EFI, Windows and Linux versions).  Here is its help someone posted:  https://ohlia.github.io/Wiki-site/wiki/Network/Intel-Ethernet-Network-Adapter-Flash-Tool-eeupdate/

Also, sometimes, powering the system off, turning off the PSU and waiting several minutes (to make certain) can cause things to recover from things like a tool complaining about the shadow RAM being corrupt/invalid.  Hopefully, no one needs to go through this.
Comment 49 Tomasz "CeDeROM" CEDRO 2026-05-18 01:35:27 UTC
Ah, thank you Sean, the same here :-)

% split -b 1M bkp.bin

-r--------   1 cederom cederom  2,0M 13 maj 21:21 bkp.bin
-rw-r--r--   1 cederom cederom  1,0M 18 maj 03:33 xaa
-rw-r--r--   1 cederom cederom  1,0M 18 maj 03:33 xab

% diff -u xaa xab
%
Comment 50 Sean Farley freebsd_committer freebsd_triage 2026-05-18 02:13:08 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #49)

You are welcome.  I wanted to share what I learned, so people would not have to spend so many hours finding out how it all worked.

Out of curiosity, did you flip the dev.igc.0.eee_control sysctl to 0 in your testing?

I disabled ASPM in the BIOS because I am testing some other issues with having NVMe drives drop sometimes (1-3 month gaps).  One time one of the HDD drives dropped too.  Supposedly, ASPM could be the culprit.  ...  It could also be the undervolting I am doing.  :D

Totally off topic, but how well does Cyberpunk 2077 run in Wine?  I have been curious about trying it out someday, especially as I do not have to worry about some client-level, anti-cheat software to ruin the day on FreeBSD.
Comment 51 Tomasz "CeDeROM" CEDRO 2026-05-18 03:07:37 UTC
Yes dev.igc.0.eee_control=0 and all works fine too.. from what I read Intel disabled power management in the firmware (probably until they sort things out) but I have no way to verify and ~1W is not a problem for me at all when things work as expected :-)

CP2077 works amazingly fine, we just do not have RTX raytracing in FreeBSD's nVidia driver, but other than that game runs amazingly smooth, far better than my previous pc with windoze natively (and older gpu) :-)

Try www.gog.com Good Old Games (and not only old) there is no drm or bs like that what you buy its yours you get offline game installers and no problematic launchers are required like in steam that blocks all of its contents for us because steam wont work in the first place :D Btw GOG was started by CD Projekt authors of Cyberpunk game :-)
Comment 52 Tomasz "CeDeROM" CEDRO 2026-08-03 01:20:00 UTC
I am done with this I226-V crap. Just ordered X540-T2 (two ports) and X540-T4 (4 ports) 10GBps second hand cards hope they work.
Comment 53 Kevin Bowling freebsd_committer freebsd_triage 2026-08-03 01:25:16 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #52)
Understandable.  It would be interesting to hear if the patches that landed in Bug#279245 have any crossover.
Comment 54 Tomasz "CeDeROM" CEDRO 2026-08-03 01:39:09 UTC
Thank you Kevin, this fix seems to be for i226 while I have i225. Looks like i225 also needs the fix :-)

https://cgit.freebsd.org/src/commit/?id=9d30fd353dd3eacb095ce98cb9c91ec015b9af64
Comment 55 commit-hook freebsd_committer freebsd_triage 2026-08-03 10:00:45 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=4a28d390f5fbae2483e88805559881b04ccf9a80

commit 4a28d390f5fbae2483e88805559881b04ccf9a80
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-08-03 09:55:15 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-08-03 10:00:00 +0000

    igc: Disable PCIe L1.2 on I225

    I225 devices can incorrectly enter L1 substates while CLKREQ# is
    asserted, both while idle and in D3.  Disable ASPM and PCI-PM L1.2 on
    I225 to prevent the resulting packet loss.

    Keep the I226 workaround ASPM-only because it addresses a separate
    traffic exit latency observation.

    PR:             265714
    MFC after:      4 days

 sys/dev/igc/if_igc.c   | 42 ++++++++++++++++++++++--------------------
 sys/dev/igc/igc_base.c | 29 +++++++++++++++++++++++++++++
 sys/dev/igc/igc_base.h |  1 +
 3 files changed, 52 insertions(+), 20 deletions(-)
Comment 56 Kevin Bowling freebsd_committer freebsd_triage 2026-08-03 10:03:03 UTC
I've expanded a related coverage to I225 matching the available errata document.  Let us know how it goes.
Comment 57 Tomasz "CeDeROM" CEDRO 2026-08-03 16:41:35 UTC
Thank you Kevin! :-)

I am using 14.4-RELEASE-p8 and releng/14.4 branch. Tried to cherry-pick changes here with this below with no success.

# git checkout releng/14.4

# git cherry-pick cfad68c5c6b9c1a203b3c3777e252048df92281d 6560ee97e8f51d5147521319dfd9d1e7afe74d34 bbf0372feeb321a5bfeff7b1e79576ab01240441 2290ea7f4311e899019fe77bf7c7775033af6b24 e35533457530bb9db655e6137c2eea790e18b97b 01e7acd38d411c78caba1c4078bb3683f586e1c2 e389a05164ccb1dd41ee8d7f09203b475322dd72 297394e995e5ea1ea9bc85e609ca116255d51e97 9d30fd353dd3eacb095ce98cb9c91ec015b9af64 cecb0f45cb83349c60514da38fddce83ad042468 4a28d390f5fbae2483e88805559881b04ccf9a80 709426551c6a3607fb5a33f5b8dbb87cfa9c8125 a108ee9138a698f212d6d6832d54e88ce6786617 8f779f159e2198c85b4fcb8685989879a9330104 

# git reflog
5c61838f749d (HEAD -> releng/14.4) HEAD@{0}: cherry-pick: igc: Add VLAN hardware filtering
2dd0fed39d00 HEAD@{1}: cherry-pick: igc: Correct hardware error statistics
b72fe84fc51b HEAD@{2}: cherry-pick: igc: Work around I225 v1 minimum IPG erratum
4b1e7e467f16 HEAD@{3}: cherry-pick: igc: Disable PCIe L1.2 on I225
34966896f03f HEAD@{4}: cherry-pick: igc: Apply ASPM L1.2 workaround to all I226 devices
450fdae4296e HEAD@{5}: cherry-pick: igc: Disable ASPM L1.2 on I226 to prevent RX stalls
9dde631ff416 HEAD@{6}: cherry-pick: igc(4): document adaptive interrupt moderation
8b8240dc63a7 HEAD@{7}: cherry-pick: igc: count TSO wire segments in the AIM counters
e48280251460 HEAD@{8}: cherry-pick: igc: use packet-size AIM
b0c5da4795be HEAD@{9}: cherry-pick: igc: synchronize interrupt moderation state
e70f6128669c HEAD@{10}: cherry-pick: igc: make AIM counter sampling coherent
6390fc3b7865 HEAD@{11}: cherry-pick: igc: fix RX accounting for multi-descriptor packets
f435040fb97e HEAD@{12}: cherry-pick: igc.4: Describe better
9972639bcf93 HEAD@{13}: cherry-pick: igc: use newly exposed RSS hash key API rather than ad-hoc hashing
3de2d29b088e (tag: release/14.4.0-p8, origin/releng/14.4) HEAD@{14}: checkout: moving from d2a5b5a86a92e86f77737273ab4b2e99da63c21d to releng/14.4

# make buildkernel
(..)
cc -target x86_64-unknown-freebsd14.4 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin -c -O2 -pipe  -fno-strict-aliasing -march=arrowlake-s -g -nostdinc  -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common    -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -MD  -MF.depend.if_igc.o -MTif_igc.o -ffile-prefix-map=/usr/src/sys=/usr/src/sys -ffile-prefix-map=/usr/obj/usr/src/amd64.amd64/sys/GENERIC=/usr/obj/usr/src/amd64.amd64/sys/GENERIC -ffile-prefix-map=/usr/obj/usr/src/amd64.amd64/tmp=/sysroot -fdebug-prefix-map=./machine=/usr/src/sys/amd64/include -fdebug-prefix-map=./x86=/usr/src/sys/x86/include -fdebug-prefix-map=./i386=/usr/src/sys/i386/include -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float  -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error=tautological-compare -Wno-error=empty-body -Wno-error=parentheses-equality -Wno-error=unused-function -Wno-error=pointer-sign -Wno-error=shift-negative-value -Wno-address-of-packed-member -Wno-format-zero-length   -mno-aes -mno-avx  -std=gnu99 -Werror /usr/src/sys/dev/igc/if_igc.c
/usr/src/sys/dev/igc/if_igc.c:1573:10: error: use of undeclared identifier 'PCIM_L1PM_CTL1_ASPM_L1_2'
 1573 |                 mask = PCIM_L1PM_CTL1_ASPM_L1_2 |
      |                        ^
/usr/src/sys/dev/igc/if_igc.c:1574:7: error: use of undeclared identifier 'PCIM_L1PM_CTL1_PCIPM_L1_2'
 1574 |                     PCIM_L1PM_CTL1_PCIPM_L1_2;
      |                     ^
/usr/src/sys/dev/igc/if_igc.c:1576:10: error: use of undeclared identifier 'PCIM_L1PM_CTL1_ASPM_L1_2'
 1576 |                 mask = PCIM_L1PM_CTL1_ASPM_L1_2;
      |                        ^
/usr/src/sys/dev/igc/if_igc.c:1583:36: error: use of undeclared identifier 'PCIR_L1PM_CTL1'
 1583 |         ctl1 = pci_read_config(dev, cap + PCIR_L1PM_CTL1, 4);
      |                                           ^
/usr/src/sys/dev/igc/if_igc.c:1585:30: error: use of undeclared identifier 'PCIR_L1PM_CTL1'
 1585 |         pci_write_config(dev, cap + PCIR_L1PM_CTL1, ctl1, 4);
      |                                     ^
5 errors generated.
*** Error code 1

Stop.
make[2]: stopped making "all" in /usr/obj/usr/src/amd64.amd64/sys/GENERIC
       67.92 real        62.26 user         5.65 sys
*** Error code 1


Looks like something else changed too in the meantime :-)

Will try to backport them into stable/14 in a moment maybe that will work :-)
Comment 58 Tomasz "CeDeROM" CEDRO 2026-08-03 16:58:36 UTC
I also have problem backporting to stable/14, sorry, do you know what additional commits do I miss? :-)

# git checkout --track origin/stable/14
branch 'stable/14' set up to track 'origin/stable/14'.
Switched to a new branch 'stable/14'

# git pull
Already up to date.

# git cherry-pick 297394e995e5ea1ea9bc85e609ca116255d51e97  9d30fd353dd3eacb095ce98cb9c91ec015b9af64  cecb0f45cb83349c60514da38fddce83ad042468  4a28d390f5fbae2483e88805559881b04ccf9a80  709426551c6a3607fb5a33f5b8dbb87cfa9c8125  a108ee9138a698f212d6d6832d54e88ce6786617  8f779f159e2198c85b4fcb8685989879a9330104
Auto-merging share/man/man4/igc.4
On branch stable/14
Your branch is up to date with 'origin/stable/14'.

You are currently cherry-picking commit 297394e995e5.
  (all conflicts fixed: run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

nothing to commit, working tree clean
The previous cherry-pick is now empty, possibly due to conflict resolution.
If you wish to commit it anyway, use:

    git commit --allow-empty

Otherwise, please use 'git cherry-pick --skip'

# git commit --allow-empty

# git cherry-pick --continue

# git commit --allow-empty
[detached HEAD e34637e3d70f] igc(4): document adaptive interrupt moderation
 Author: Kevin Bowling <kbowling@FreeBSD.org>
 Date: Sat Jul 25 17:10:38 2026 -0700
hexagon# git cherry-pick --continue
Auto-merging sys/dev/igc/if_igc.c
[detached HEAD 1a02a700d43e] igc: Disable ASPM L1.2 on I226 to prevent RX stalls
 Author: Michael Adler <madler@tapil.com>
 Date: Thu Jul 9 13:02:37 2026 -0400
 3 files changed, 66 insertions(+)
[detached HEAD 2222df13f495] igc: Apply ASPM L1.2 workaround to all I226 devices
 Author: Michael Adler <madler@tapil.com>
 Date: Fri Jul 31 16:54:00 2026 -0700
 1 file changed, 5 insertions(+)
Auto-merging sys/dev/igc/if_igc.c
[detached HEAD 64edcfc0b200] igc: Disable PCIe L1.2 on I225
 Author: Kevin Bowling <kbowling@FreeBSD.org>
 Date: Mon Aug 3 02:55:15 2026 -0700
 3 files changed, 52 insertions(+), 20 deletions(-)
Auto-merging sys/dev/igc/if_igc.c
[detached HEAD 013eaf9d4559] igc: Work around I225 v1 minimum IPG erratum
 Author: Kevin Bowling <kbowling@FreeBSD.org>
 Date: Mon Aug 3 03:28:49 2026 -0700
 2 files changed, 30 insertions(+)
Auto-merging sys/dev/igc/if_igc.c
[detached HEAD 89996d34bf2a] igc: Correct hardware error statistics
 Author: Kevin Bowling <kbowling@FreeBSD.org>
 Date: Mon Aug 3 03:31:36 2026 -0700
 4 files changed, 17 insertions(+), 3 deletions(-)
Auto-merging share/man/man4/igc.4
CONFLICT (content): Merge conflict in share/man/man4/igc.4
Auto-merging sys/dev/igc/if_igc.c
error: could not apply 8f779f159e21... igc: Add VLAN hardware filtering
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config set advice.mergeConflict false"

# git status
HEAD detached from origin/stable/14
You are currently cherry-picking commit 8f779f159e21.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   sys/dev/igc/if_igc.c
	modified:   sys/dev/igc/if_igc.h

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   share/man/man4/igc.4

# vim share/man/man4/igc.4
# git add share/man/man4/igc.4
# git cherry-pick --continue

4034a2a6dd86 (HEAD) HEAD@{0}: commit (cherry-pick): igc: Add VLAN hardware filtering
89996d34bf2a HEAD@{1}: cherry-pick: igc: Correct hardware error statistics
013eaf9d4559 HEAD@{2}: cherry-pick: igc: Work around I225 v1 minimum IPG erratum
64edcfc0b200 HEAD@{3}: cherry-pick: igc: Disable PCIe L1.2 on I225
2222df13f495 HEAD@{4}: cherry-pick: igc: Apply ASPM L1.2 workaround to all I226 devices
1a02a700d43e HEAD@{5}: cherry-pick: igc: Disable ASPM L1.2 on I226 to prevent RX stalls
e34637e3d70f HEAD@{6}: commit (cherry-pick): igc(4): document adaptive interrupt moderation
0194bbf16ff9 HEAD@{7}: commit (cherry-pick): igc(4): document adaptive interrupt moderation
58c826d72918 (origin/stable/14, stable/14) HEAD@{8}: checkout: moving from stable/14 to origin/stable/14

# make buildkernel
(..)
cc -target x86_64-unknown-freebsd14.5 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin -c -O2 -pipe  -fno-strict-aliasing -march=arrowlake-s -g -nostdinc  -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common    -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -MD  -MF.depend.if_igc.o -MTif_igc.o -ffile-prefix-map=/usr/src/sys=/usr/src/sys -ffile-prefix-map=/usr/obj/usr/src/amd64.amd64/sys/GENERIC=/usr/obj/usr/src/amd64.amd64/sys/GENERIC -ffile-prefix-map=/usr/obj/usr/src/amd64.amd64/tmp=/sysroot -fdebug-prefix-map=./machine=/usr/src/sys/amd64/include -fdebug-prefix-map=./x86=/usr/src/sys/x86/include -fdebug-prefix-map=./i386=/usr/src/sys/i386/include -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float  -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector-strong -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error=tautological-compare -Wno-error=empty-body -Wno-error=parentheses-equality -Wno-error=unused-function -Wno-error=pointer-sign -Wno-error=shift-negative-value -Wno-address-of-packed-member -Wno-format-zero-length   -mno-aes -mno-avx  -std=gnu99 -Werror /usr/src/sys/dev/igc/if_igc.c
/usr/src/sys/dev/igc/if_igc.c:1574:10: error: use of undeclared identifier 'PCIM_L1PM_CTL1_ASPM_L1_2'
 1574 |                 mask = PCIM_L1PM_CTL1_ASPM_L1_2 |
      |                        ^
/usr/src/sys/dev/igc/if_igc.c:1575:7: error: use of undeclared identifier 'PCIM_L1PM_CTL1_PCIPM_L1_2'
 1575 |                     PCIM_L1PM_CTL1_PCIPM_L1_2;
      |                     ^
/usr/src/sys/dev/igc/if_igc.c:1577:10: error: use of undeclared identifier 'PCIM_L1PM_CTL1_ASPM_L1_2'
 1577 |                 mask = PCIM_L1PM_CTL1_ASPM_L1_2;
      |                        ^
/usr/src/sys/dev/igc/if_igc.c:1584:36: error: use of undeclared identifier 'PCIR_L1PM_CTL1'
 1584 |         ctl1 = pci_read_config(dev, cap + PCIR_L1PM_CTL1, 4);
      |                                           ^
/usr/src/sys/dev/igc/if_igc.c:1586:30: error: use of undeclared identifier 'PCIR_L1PM_CTL1'
 1586 |         pci_write_config(dev, cap + PCIR_L1PM_CTL1, ctl1, 4);
      |                                     ^
5 errors generated.
*** Error code 1

Stop.
make[2]: stopped making "all" in /usr/obj/usr/src/amd64.amd64/sys/GENERIC
       69.51 real        63.22 user         6.30 sys
*** Error code 1
Comment 59 Tomasz "CeDeROM" CEDRO 2026-08-03 18:27:59 UTC
Playing around some more, here is a dedicated branch in case someone want to test on top of existing 14.4-RELEASE-p8:

https://github.com/cederom/freebsd-src/tree/cederom-releng-14.4.8-igc_test_20260803

So far this is:

# git log --oneline -17
18774c90fb02 (HEAD -> cederom-releng-14.4.8-igc_test_20260803, cederom/cederom-releng-14.4.8-igc_test_20260803) igc: Add VLAN hardware filtering
c1bc2da741d5 igc: Correct hardware error statistics
e80624657a23 igc: Work around I225 v1 minimum IPG erratum
8429f0391c11 igc: Disable PCIe L1.2 on I225
ea774409c9ca igc: Apply ASPM L1.2 workaround to all I226 devices
df561a9ea5f9 igc: Disable ASPM L1.2 on I226 to prevent RX stalls
4472d239ff5b igc(4): document adaptive interrupt moderation
53408cbd10c1 igc: count TSO wire segments in the AIM counters
f94e9349b389 igc: use packet-size AIM
fe2c8e215ccf igc: synchronize interrupt moderation state
fdf2a8ad9699 igc: make AIM counter sampling coherent
dc0fd5a3b87b igc: fix RX accounting for multi-descriptor packets
d6f358d9412c igc.4: Describe better
a3a185dba982 igc: use newly exposed RSS hash key API rather than ad-hoc hashing
518828a76cef pci: Add L1 PM definitions
d9b8b38e449f rss: Enable portions of RSS globally to enable symmetric hashing
3de2d29b088e (tag: release/14.4.0-p8, origin/releng/14.4, cederom/releng/14.4) Add UPDATING entries and bump version
a830cbd7bbef HEAD@{14}: cherry-pick: pci: Add L1 PM definitions
Comment 60 Tomasz "CeDeROM" CEDRO 2026-08-04 00:38:23 UTC
Builds fine now, some symbols are added / changed thus reboot was necessary, works fine so far, need to work for some time (days) with that update to see the any change, the path for stable/14 and releng/14.4 is reported, thank you Kevin!! :-)

# uname -a
FreeBSD hexagon 14.4-RELEASE-p8 FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260803-n273770-18774c90fb02 GENERIC amd64
Comment 61 Tomasz "CeDeROM" CEDRO 2026-08-04 17:16:15 UTC
So far so good, link is up :-)

I also used Mike's script from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265714#c0, could not cause link drops on purpose, but not sure if I could do that before a fix (can test if needed).

ps/2: Both X504-T2 and X504-T4 just arrived (perfectly synced with IGC fixes heh), shows up as ix<n>, need to find a way to firmware update.. but for now I am still working on the IGC :-)
Comment 62 mike 2026-08-05 12:48:25 UTC
I did some stress testing with a Baytrails chipset box.

# pciconf -lvcb igc2
igc2@pci0:3:0:0:        class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 32, base rxd0a00000, size 1048576, enabled
    bar   [1c] = type Memory, range 32, base rxd0b00000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR NS
                 max read 512
                 link x1(x1) speed 5.0(5.0) ASPM L1(L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 00e023ffff5d8a8a
    ecap 0018[1c0] = LTR 1
    ecap 001f[1f0] = Precision Time Measurement 1
    ecap 001e[1e0] = L1 PM Substates 1
14.5-PRERELEASE stable/14-aa8ffe6fbac7

sysctl -a dev.igc.2
dev.igc.2.wake: 0
dev.igc.2.interrupts.rx_desc_min_thresh: 0
dev.igc.2.interrupts.asserts: 903870504
dev.igc.2.mac_stats.tso_txd: 432492322
dev.igc.2.mac_stats.tx_frames_1024_1522: 5679253623
dev.igc.2.mac_stats.tx_frames_512_1023: 83184984
dev.igc.2.mac_stats.tx_frames_256_511: 275787
dev.igc.2.mac_stats.tx_frames_128_255: 334922
dev.igc.2.mac_stats.tx_frames_65_127: 2039817773
dev.igc.2.mac_stats.tx_frames_64: 93486
dev.igc.2.mac_stats.mcast_pkts_txd: 0
dev.igc.2.mac_stats.bcast_pkts_txd: 0
dev.igc.2.mac_stats.good_pkts_txd: 7802960572
dev.igc.2.mac_stats.total_pkts_txd: 7802960574
dev.igc.2.mac_stats.good_octets_txd: 8831858404092
dev.igc.2.mac_stats.good_octets_recvd: 6186583531054
dev.igc.2.mac_stats.rx_frames_1024_1522: 4056500391
dev.igc.2.mac_stats.rx_frames_512_1023: 32564
dev.igc.2.mac_stats.rx_frames_256_511: 14563
dev.igc.2.mac_stats.rx_frames_128_255: 8743
dev.igc.2.mac_stats.rx_frames_65_127: 411163341
dev.igc.2.mac_stats.rx_frames_64: 108345
dev.igc.2.mac_stats.mcast_pkts_recvd: 0
dev.igc.2.mac_stats.bcast_pkts_recvd: 2
dev.igc.2.mac_stats.good_pkts_recvd: 4467827947
dev.igc.2.mac_stats.total_pkts_recvd: 4467827965
dev.igc.2.mac_stats.mgmt_pkts_txd: 0
dev.igc.2.mac_stats.mgmt_pkts_drop: 0
dev.igc.2.mac_stats.mgmt_pkts_recvd: 0
dev.igc.2.mac_stats.unsupported_fc_recvd: 0
dev.igc.2.mac_stats.xoff_txd: 0
dev.igc.2.mac_stats.xoff_recvd: 0
dev.igc.2.mac_stats.xon_txd: 0
dev.igc.2.mac_stats.xon_recvd: 0
dev.igc.2.mac_stats.alignment_errs: 0
dev.igc.2.mac_stats.crc_errs: 0
dev.igc.2.mac_stats.recv_errs: 0
dev.igc.2.mac_stats.recv_jabber: 0
dev.igc.2.mac_stats.recv_oversize: 0
dev.igc.2.mac_stats.recv_fragmented: 0
dev.igc.2.mac_stats.recv_undersize: 0
dev.igc.2.mac_stats.recv_no_buff: 0
dev.igc.2.mac_stats.recv_length_errors: 0
dev.igc.2.mac_stats.missed_packets: 0
dev.igc.2.mac_stats.defer_count: 0
dev.igc.2.mac_stats.sequence_errors: 0
dev.igc.2.mac_stats.symbol_errors: 0
dev.igc.2.mac_stats.collision_count: 0
dev.igc.2.mac_stats.late_coll: 0
dev.igc.2.mac_stats.multiple_coll: 0
dev.igc.2.mac_stats.single_coll: 0
dev.igc.2.mac_stats.excess_coll: 0
dev.igc.2.queue_rx_3.rx_irq: 0
dev.igc.2.queue_rx_3.rxd_tail: 602
dev.igc.2.queue_rx_3.rxd_head: 603
dev.igc.2.queue_rx_3.interrupt_rate: 20000
dev.igc.2.queue_rx_2.rx_irq: 0
dev.igc.2.queue_rx_2.rxd_tail: 158
dev.igc.2.queue_rx_2.rxd_head: 159
dev.igc.2.queue_rx_2.interrupt_rate: 20000
dev.igc.2.queue_rx_1.rx_irq: 0
dev.igc.2.queue_rx_1.rxd_tail: 75
dev.igc.2.queue_rx_1.rxd_head: 968
dev.igc.2.queue_rx_1.interrupt_rate: 5208
dev.igc.2.queue_rx_0.rx_irq: 0
dev.igc.2.queue_rx_0.rxd_tail: 147
dev.igc.2.queue_rx_0.rxd_head: 148
dev.igc.2.queue_rx_0.interrupt_rate: 20000
dev.igc.2.queue_tx_3.tx_irq: 0
dev.igc.2.queue_tx_3.txd_tail: 770
dev.igc.2.queue_tx_3.txd_head: 770
dev.igc.2.queue_tx_3.interrupt_rate: 20000
dev.igc.2.queue_tx_2.tx_irq: 0
dev.igc.2.queue_tx_2.txd_tail: 159
dev.igc.2.queue_tx_2.txd_head: 159
dev.igc.2.queue_tx_2.interrupt_rate: 20000
dev.igc.2.queue_tx_1.tx_irq: 0
dev.igc.2.queue_tx_1.txd_tail: 795
dev.igc.2.queue_tx_1.txd_head: 801
dev.igc.2.queue_tx_1.interrupt_rate: 5208
dev.igc.2.queue_tx_0.tx_irq: 0
dev.igc.2.queue_tx_0.txd_tail: 284
dev.igc.2.queue_tx_0.txd_head: 284
dev.igc.2.queue_tx_0.interrupt_rate: 20000
dev.igc.2.fc_low_water: 32752
dev.igc.2.fc_high_water: 32768
dev.igc.2.rx_control: 71335938
dev.igc.2.device_control: 1478231617
dev.igc.2.watchdog_timeouts: 0
dev.igc.2.rx_overruns: 0
dev.igc.2.link_irq: 8
dev.igc.2.dropped: 0
dev.igc.2.eee_control: 1
dev.igc.2.tso_tcp_flags_mask_last_segment: 3967
dev.igc.2.tso_tcp_flags_mask_middle_segment: 3958
dev.igc.2.tso_tcp_flags_mask_first_segment: 4086
dev.igc.2.dmac: 0
dev.igc.2.rs_dump: 0
dev.igc.2.reg_dump: General Registers
        CTRL     581c0641
        STATUS   00780683
        CTRL_EXIT        100000c0

Interrupt Registers
        ICR      00000081

RX Registers
        RCTL     04408002
        RDLEN    00004000
        RDH      00000094
        RDT      00000093
        RXDCTL   02040808
        RDBAL    02988000
        RDBAH    00000000

TX Registers
        TCTL     a503f0fa
        TDBAL    0294a000
        TDBAH    00000000
        TDLEN    00004000
        TDH      0000011c
        TDT      0000011c
        TXDCTL   0201011f
        TDFH     00000000
        TDFT     00000000
        TDFHS    00000000
        TDFPC    00000000


dev.igc.2.fc: 3
dev.igc.2.debug: -1
dev.igc.2.fw_version: EEPROM V2.17-0 eTrack 0x80000303
dev.igc.2.enable_aim: 1
dev.igc.2.nvm: -1
dev.igc.2.iflib.rxq3.rxq_fl0.buf_size: 2048
dev.igc.2.iflib.rxq3.rxq_fl0.credits: 1023
dev.igc.2.iflib.rxq3.rxq_fl0.cidx: 603
dev.igc.2.iflib.rxq3.rxq_fl0.pidx: 602
dev.igc.2.iflib.rxq3.cpu: 3
dev.igc.2.iflib.rxq2.rxq_fl0.buf_size: 2048
dev.igc.2.iflib.rxq2.rxq_fl0.credits: 1023
dev.igc.2.iflib.rxq2.rxq_fl0.cidx: 159
dev.igc.2.iflib.rxq2.rxq_fl0.pidx: 158
dev.igc.2.iflib.rxq2.cpu: 2
dev.igc.2.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igc.2.iflib.rxq1.rxq_fl0.credits: 1023
dev.igc.2.iflib.rxq1.rxq_fl0.cidx: 497
dev.igc.2.iflib.rxq1.rxq_fl0.pidx: 507
dev.igc.2.iflib.rxq1.cpu: 1
dev.igc.2.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igc.2.iflib.rxq0.rxq_fl0.credits: 1023
dev.igc.2.iflib.rxq0.rxq_fl0.cidx: 148
dev.igc.2.iflib.rxq0.rxq_fl0.pidx: 147
dev.igc.2.iflib.rxq0.cpu: 0
dev.igc.2.iflib.txq3.r_abdications: 2
dev.igc.2.iflib.txq3.r_restarts: 6633909
dev.igc.2.iflib.txq3.r_stalls: 6633909
dev.igc.2.iflib.txq3.r_starts: 531849323
dev.igc.2.iflib.txq3.r_drops: 0
dev.igc.2.iflib.txq3.r_enqueues: 577551561
dev.igc.2.iflib.txq3.ring_state: pidx_head: 1225 pidx_tail: 1225 cidx: 1225 state: IDLE
dev.igc.2.iflib.txq3.txq_cleaned: 1653054166
dev.igc.2.iflib.txq3.txq_processed: 1653054206
dev.igc.2.iflib.txq3.txq_in_use: 44
dev.igc.2.iflib.txq3.txq_cidx_processed: 766
dev.igc.2.iflib.txq3.txq_cidx: 726
dev.igc.2.iflib.txq3.txq_pidx: 770
dev.igc.2.iflib.txq3.no_tx_dma_setup: 0
dev.igc.2.iflib.txq3.txd_encap_efbig: 0
dev.igc.2.iflib.txq3.tx_map_failed: 0
dev.igc.2.iflib.txq3.no_desc_avail: 0
dev.igc.2.iflib.txq3.mbuf_defrag_failed: 0
dev.igc.2.iflib.txq3.m_pullups: 0
dev.igc.2.iflib.txq3.mbuf_defrag: 0
dev.igc.2.iflib.txq3.cpu: 3
dev.igc.2.iflib.txq2.r_abdications: 2
dev.igc.2.iflib.txq2.r_restarts: 7123105
dev.igc.2.iflib.txq2.r_stalls: 7123105
dev.igc.2.iflib.txq2.r_starts: 573192125
dev.igc.2.iflib.txq2.r_drops: 0
dev.igc.2.iflib.txq2.r_enqueues: 622556978
dev.igc.2.iflib.txq2.ring_state: pidx_head: 1842 pidx_tail: 1842 cidx: 1842 state: IDLE
dev.igc.2.iflib.txq2.txq_cleaned: 1795436659
dev.igc.2.iflib.txq2.txq_processed: 1795436699
dev.igc.2.iflib.txq2.txq_in_use: 44
dev.igc.2.iflib.txq2.txq_cidx_processed: 155
dev.igc.2.iflib.txq2.txq_cidx: 115
dev.igc.2.iflib.txq2.txq_pidx: 159
dev.igc.2.iflib.txq2.no_tx_dma_setup: 0
dev.igc.2.iflib.txq2.txd_encap_efbig: 0
dev.igc.2.iflib.txq2.tx_map_failed: 0
dev.igc.2.iflib.txq2.no_desc_avail: 0
dev.igc.2.iflib.txq2.mbuf_defrag_failed: 0
dev.igc.2.iflib.txq2.m_pullups: 0
dev.igc.2.iflib.txq2.mbuf_defrag: 0
dev.igc.2.iflib.txq2.cpu: 2
dev.igc.2.iflib.txq1.r_abdications: 0
dev.igc.2.iflib.txq1.r_restarts: 5928460
dev.igc.2.iflib.txq1.r_stalls: 5928460
dev.igc.2.iflib.txq1.r_starts: 592070605
dev.igc.2.iflib.txq1.r_drops: 0
dev.igc.2.iflib.txq1.r_enqueues: 630706440
dev.igc.2.iflib.txq1.ring_state: pidx_head: 0266 pidx_tail: 0266 cidx: 0266 state: IDLE
dev.igc.2.iflib.txq1.txq_cleaned: 1848925499
dev.igc.2.iflib.txq1.txq_processed: 1848925543
dev.igc.2.iflib.txq1.txq_in_use: 42
dev.igc.2.iflib.txq1.txq_cidx_processed: 367
dev.igc.2.iflib.txq1.txq_cidx: 335
dev.igc.2.iflib.txq1.txq_pidx: 381
dev.igc.2.iflib.txq1.no_tx_dma_setup: 0
dev.igc.2.iflib.txq1.txd_encap_efbig: 0
dev.igc.2.iflib.txq1.tx_map_failed: 0
dev.igc.2.iflib.txq1.no_desc_avail: 0
dev.igc.2.iflib.txq1.mbuf_defrag_failed: 0
dev.igc.2.iflib.txq1.m_pullups: 0
dev.igc.2.iflib.txq1.mbuf_defrag: 0
dev.igc.2.iflib.txq1.cpu: 1
dev.igc.2.iflib.txq0.r_abdications: 0
dev.igc.2.iflib.txq0.r_restarts: 7519491
dev.igc.2.iflib.txq0.r_stalls: 7519491
dev.igc.2.iflib.txq0.r_starts: 594963050
dev.igc.2.iflib.txq0.r_drops: 0
dev.igc.2.iflib.txq0.r_enqueues: 647338399
dev.igc.2.iflib.txq0.ring_state: pidx_head: 0415 pidx_tail: 0415 cidx: 0415 state: IDLE
dev.igc.2.iflib.txq0.txq_cleaned: 1880386948
dev.igc.2.iflib.txq0.txq_processed: 1880386988
dev.igc.2.iflib.txq0.txq_in_use: 920
dev.igc.2.iflib.txq0.txq_cidx_processed: 428
dev.igc.2.iflib.txq0.txq_cidx: 388
dev.igc.2.iflib.txq0.txq_pidx: 284
dev.igc.2.iflib.txq0.no_tx_dma_setup: 0
dev.igc.2.iflib.txq0.txd_encap_efbig: 0
dev.igc.2.iflib.txq0.tx_map_failed: 0
dev.igc.2.iflib.txq0.no_desc_avail: 0
dev.igc.2.iflib.txq0.mbuf_defrag_failed: 0
dev.igc.2.iflib.txq0.m_pullups: 0
dev.igc.2.iflib.txq0.mbuf_defrag: 0
dev.igc.2.iflib.txq0.cpu: 0
dev.igc.2.iflib.override_nrxds: 0
dev.igc.2.iflib.override_ntxds: 0
dev.igc.2.iflib.allocated_msix_vectors: 5
dev.igc.2.iflib.use_extra_msix_vectors: 0
dev.igc.2.iflib.use_logical_cores: 0
dev.igc.2.iflib.separate_txrx: 0
dev.igc.2.iflib.core_offset: 0
dev.igc.2.iflib.tx_abdicate: 0
dev.igc.2.iflib.rx_budget: 0
dev.igc.2.iflib.disable_msix: 0
dev.igc.2.iflib.override_qs_enable: 0
dev.igc.2.iflib.override_nrxqs: 0
dev.igc.2.iflib.override_ntxqs: 0
dev.igc.2.iflib.driver_version: 1
dev.igc.2.%iommu: rid=0x300
dev.igc.2.%parent: pci3
dev.igc.2.%pnpinfo: vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.igc.2.%location: slot=0 function=0 dbsf=pci0:3:0:0 handle=\_SB_.PCI0.RP03.PXSX
dev.igc.2.%driver: igc
dev.igc.2.%desc: Intel(R) Ethernet Controller I226-V

I was sending a recving from a faster box (n150 vs J1900) as well as had pf and argus enabled for half of the 12hr test so it could not go at full line rate. But it didnt drop the link so that seems pretty good so far.  
I have a number of I225's I will test with too once the changes are MFCd. I see there are a few other changes coming soon, so I will retest once those are in 14 stable.
Comment 63 Tomasz "CeDeROM" CEDRO 2026-08-05 15:18:23 UTC
Over 24h testing with iperf3 in the background, vbox virtual machines on/off, and standard workflow, no problem so far! :-)

Would be great if fixes landed into stable/14 and releng/14.4 :-)

THANK YOU! :-)
Comment 64 Tomasz "CeDeROM" CEDRO 2026-08-06 23:49:21 UTC
I just got the IGC network interface stall. No particular reason or high load. VBox working in the background. Had to manually restart.

# uname -a
FreeBSD hexagon 14.4-RELEASE-p8 FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260803-n273770-18774c90fb02 GENERIC amd64


# sysctl -a | grep igc
kern.version: FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260803-n273770-18774c90fb02 GENERIC
FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260803-n273770-18774c90fb02 GENERIC amd64
[1] igc0: <Intel(R) Ethernet Controller I226-V> mem 0xb2500000-0xb25fffff,0xb2600000-0xb2603fff at device 0.0 on pci7
[1] igc0: EEPROM V2.32-0 eTrack 0x80000425
[1] igc0: Using 1024 TX descriptors and 1024 RX descriptors
[1] igc0: Using 4 RX queues 4 TX queues
[1] igc0: Using MSI-X interrupts with 5 vectors
<6>[1] igc0: Ethernet address: XXX
<6>[1] igc0: netmap queues/slots: TX 4/1024, RX 4/1024
<6>[30] igc0: link state changed to UP
<6>[30] igc0: link state changed to DOWN
<118>[30] Starting Network: lo0 igc0 ix0 ix1.
<118>[30] igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
<6>[34] igc0: link state changed to UP
<118>[37] DHCPDISCOVER on igc0 to 255.255.255.255 port 67 interval 3
<118>[37] DHCPREQUEST on igc0 to 255.255.255.255 port 67
device	igc
kern.wait_dequeue_sigchld: 1
irq151: igc0:rxq0:325 @cpu0(domain0): 1068347
irq152: igc0:rxq1:327 @cpu1(domain0): 1900924
irq153: igc0:rxq2:329 @cpu2(domain0): 678528
irq154: igc0:rxq3:331 @cpu3(domain0): 623502
irq155: igc0:aq:333 @cpu0(domain0): 2
hw.igc.max_interrupt_rate: 16000
hw.igc.enable_aim: 1
hw.igc.eee_setting: 1
hw.igc.sbp: 1
hw.igc.smart_pwr_down: 0
hw.igc.disable_crc_stripping: 0
dev.igc.0.wake: 0
dev.igc.0.interrupts.rx_desc_min_thresh: 0
dev.igc.0.interrupts.asserts: 4278433
dev.igc.0.mac_stats.tso_txd: 6304
dev.igc.0.mac_stats.tx_frames_1024_1522: 83542
dev.igc.0.mac_stats.tx_frames_512_1023: 19259
dev.igc.0.mac_stats.tx_frames_256_511: 38606
dev.igc.0.mac_stats.tx_frames_128_255: 48027
dev.igc.0.mac_stats.tx_frames_65_127: 2143703
dev.igc.0.mac_stats.tx_frames_64: 6540
dev.igc.0.mac_stats.mcast_pkts_txd: 0
dev.igc.0.mac_stats.bcast_pkts_txd: 83
dev.igc.0.mac_stats.host_tx_discarded: 0
dev.igc.0.mac_stats.good_pkts_txd: 2339677
dev.igc.0.mac_stats.total_pkts_txd: 2339677
dev.igc.0.mac_stats.good_octets_txd: 299317877
dev.igc.0.mac_stats.good_octets_recvd: 11363569459
dev.igc.0.mac_stats.rx_frames_1024_1522: 7575136
dev.igc.0.mac_stats.rx_frames_512_1023: 387645
dev.igc.0.mac_stats.rx_frames_256_511: 205117
dev.igc.0.mac_stats.rx_frames_128_255: 105652
dev.igc.0.mac_stats.rx_frames_65_127: 214364
dev.igc.0.mac_stats.rx_frames_64: 1199
dev.igc.0.mac_stats.mcast_pkts_recvd: 0
dev.igc.0.mac_stats.bcast_pkts_recvd: 3669
dev.igc.0.mac_stats.good_pkts_recvd: 8489113
dev.igc.0.mac_stats.total_pkts_recvd: 8497819
dev.igc.0.mac_stats.mgmt_pkts_txd: 0
dev.igc.0.mac_stats.mgmt_pkts_drop: 0
dev.igc.0.mac_stats.mgmt_pkts_recvd: 0
dev.igc.0.mac_stats.unsupported_fc_recvd: 0
dev.igc.0.mac_stats.xoff_txd: 0
dev.igc.0.mac_stats.xoff_recvd: 0
dev.igc.0.mac_stats.xon_txd: 0
dev.igc.0.mac_stats.xon_recvd: 0
dev.igc.0.mac_stats.alignment_errs: 0
dev.igc.0.mac_stats.crc_errs: 0
dev.igc.0.mac_stats.recv_error_count: 0
dev.igc.0.mac_stats.recv_errs: 0
dev.igc.0.mac_stats.recv_jabber: 0
dev.igc.0.mac_stats.recv_oversize: 0
dev.igc.0.mac_stats.recv_fragmented: 0
dev.igc.0.mac_stats.recv_undersize: 0
dev.igc.0.mac_stats.recv_no_buff: 0
dev.igc.0.mac_stats.recv_length_errors: 0
dev.igc.0.mac_stats.missed_packets: 322
dev.igc.0.mac_stats.defer_count: 0
dev.igc.0.mac_stats.sequence_errors: 0
dev.igc.0.mac_stats.symbol_errors: 0
dev.igc.0.mac_stats.collision_count: 0
dev.igc.0.mac_stats.late_coll: 0
dev.igc.0.mac_stats.multiple_coll: 0
dev.igc.0.mac_stats.single_coll: 0
dev.igc.0.mac_stats.excess_coll: 0
dev.igc.0.queue_rx_3.rx_irq: 0
dev.igc.0.queue_rx_3.rxd_tail: 448
dev.igc.0.queue_rx_3.rxd_head: 449
dev.igc.0.queue_rx_3.interrupt_rate: 20000
dev.igc.0.queue_rx_2.rx_irq: 0
dev.igc.0.queue_rx_2.rxd_tail: 114
dev.igc.0.queue_rx_2.rxd_head: 115
dev.igc.0.queue_rx_2.interrupt_rate: 20000
dev.igc.0.queue_rx_1.rx_irq: 0
dev.igc.0.queue_rx_1.rxd_tail: 118
dev.igc.0.queue_rx_1.rxd_head: 119
dev.igc.0.queue_rx_1.interrupt_rate: 20000
dev.igc.0.queue_rx_0.rx_irq: 0
dev.igc.0.queue_rx_0.rxd_tail: 83
dev.igc.0.queue_rx_0.rxd_head: 83
dev.igc.0.queue_rx_0.interrupt_rate: 5291
dev.igc.0.queue_tx_3.tx_irq: 0
dev.igc.0.queue_tx_3.txd_tail: 395
dev.igc.0.queue_tx_3.txd_head: 395
dev.igc.0.queue_tx_3.interrupt_rate: 20000
dev.igc.0.queue_tx_2.tx_irq: 0
dev.igc.0.queue_tx_2.txd_tail: 939
dev.igc.0.queue_tx_2.txd_head: 939
dev.igc.0.queue_tx_2.interrupt_rate: 20000
dev.igc.0.queue_tx_1.tx_irq: 0
dev.igc.0.queue_tx_1.txd_tail: 851
dev.igc.0.queue_tx_1.txd_head: 851
dev.igc.0.queue_tx_1.interrupt_rate: 20000
dev.igc.0.queue_tx_0.tx_irq: 0
dev.igc.0.queue_tx_0.txd_tail: 418
dev.igc.0.queue_tx_0.txd_head: 418
dev.igc.0.queue_tx_0.interrupt_rate: 5291
dev.igc.0.fc_low_water: 32752
dev.igc.0.fc_high_water: 32768
dev.igc.0.rx_control: 71598082
dev.igc.0.device_control: 1209796161
dev.igc.0.watchdog_timeouts: 0
dev.igc.0.rx_overruns: 0
dev.igc.0.link_irq: 2
dev.igc.0.dropped: 0
dev.igc.0.eee_control: 1
dev.igc.0.tso_tcp_flags_mask_last_segment: 3967
dev.igc.0.tso_tcp_flags_mask_middle_segment: 3958
dev.igc.0.tso_tcp_flags_mask_first_segment: 4086
dev.igc.0.dmac: 0
dev.igc.0.rs_dump: 0
dev.igc.0.reg_dump: General Registers
dev.igc.0.fc: 3
dev.igc.0.debug: -1
dev.igc.0.fw_version: EEPROM V2.32-0 eTrack 0x80000425
dev.igc.0.enable_aim: 1
dev.igc.0.nvm: -1
dev.igc.0.iflib.rxq3.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq3.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq3.rxq_fl0.cidx: 449
dev.igc.0.iflib.rxq3.rxq_fl0.pidx: 448
dev.igc.0.iflib.rxq3.cpu: 3
dev.igc.0.iflib.rxq2.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq2.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq2.rxq_fl0.cidx: 115
dev.igc.0.iflib.rxq2.rxq_fl0.pidx: 114
dev.igc.0.iflib.rxq2.cpu: 2
dev.igc.0.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq1.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq1.rxq_fl0.cidx: 119
dev.igc.0.iflib.rxq1.rxq_fl0.pidx: 118
dev.igc.0.iflib.rxq1.cpu: 1
dev.igc.0.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igc.0.iflib.rxq0.rxq_fl0.credits: 1023
dev.igc.0.iflib.rxq0.rxq_fl0.cidx: 84
dev.igc.0.iflib.rxq0.rxq_fl0.pidx: 83
dev.igc.0.iflib.rxq0.cpu: 0
dev.igc.0.iflib.txq3.r_abdications: 0
dev.igc.0.iflib.txq3.r_restarts: 0
dev.igc.0.iflib.txq3.r_stalls: 0
dev.igc.0.iflib.txq3.r_starts: 328999
dev.igc.0.iflib.txq3.r_drops: 0
dev.igc.0.iflib.txq3.r_enqueues: 329013
dev.igc.0.iflib.txq3.ring_state: pidx_head: 1333 pidx_tail: 1333 cidx: 1333 state: IDLE
dev.igc.0.iflib.txq3.txq_cleaned: 680289
dev.igc.0.iflib.txq3.txq_processed: 680329
dev.igc.0.iflib.txq3.txq_in_use: 42
dev.igc.0.iflib.txq3.txq_cidx_processed: 393
dev.igc.0.iflib.txq3.txq_cidx: 353
dev.igc.0.iflib.txq3.txq_pidx: 395
dev.igc.0.iflib.txq3.no_tx_dma_setup: 0
dev.igc.0.iflib.txq3.txd_encap_efbig: 0
dev.igc.0.iflib.txq3.tx_map_failed: 0
dev.igc.0.iflib.txq3.no_desc_avail: 0
dev.igc.0.iflib.txq3.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq3.m_pullups: 0
dev.igc.0.iflib.txq3.mbuf_defrag: 0
dev.igc.0.iflib.txq3.cpu: 3
dev.igc.0.iflib.txq2.r_abdications: 0
dev.igc.0.iflib.txq2.r_restarts: 0
dev.igc.0.iflib.txq2.r_stalls: 0
dev.igc.0.iflib.txq2.r_starts: 339320
dev.igc.0.iflib.txq2.r_drops: 0
dev.igc.0.iflib.txq2.r_enqueues: 339340
dev.igc.0.iflib.txq2.ring_state: pidx_head: 1420 pidx_tail: 1420 cidx: 1420 state: IDLE
dev.igc.0.iflib.txq2.txq_cleaned: 713600
dev.igc.0.iflib.txq2.txq_processed: 713640
dev.igc.0.iflib.txq2.txq_in_use: 43
dev.igc.0.iflib.txq2.txq_cidx_processed: 936
dev.igc.0.iflib.txq2.txq_cidx: 896
dev.igc.0.iflib.txq2.txq_pidx: 939
dev.igc.0.iflib.txq2.no_tx_dma_setup: 0
dev.igc.0.iflib.txq2.txd_encap_efbig: 0
dev.igc.0.iflib.txq2.tx_map_failed: 0
dev.igc.0.iflib.txq2.no_desc_avail: 0
dev.igc.0.iflib.txq2.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq2.m_pullups: 0
dev.igc.0.iflib.txq2.mbuf_defrag: 0
dev.igc.0.iflib.txq2.cpu: 2
dev.igc.0.iflib.txq1.r_abdications: 0
dev.igc.0.iflib.txq1.r_restarts: 0
dev.igc.0.iflib.txq1.r_stalls: 0
dev.igc.0.iflib.txq1.r_starts: 955948
dev.igc.0.iflib.txq1.r_drops: 0
dev.igc.0.iflib.txq1.r_enqueues: 955983
dev.igc.0.iflib.txq1.ring_state: pidx_head: 1615 pidx_tail: 1615 cidx: 1615 state: IDLE
dev.igc.0.iflib.txq1.txq_cleaned: 1941287
dev.igc.0.iflib.txq1.txq_processed: 1941327
dev.igc.0.iflib.txq1.txq_in_use: 44
dev.igc.0.iflib.txq1.txq_cidx_processed: 847
dev.igc.0.iflib.txq1.txq_cidx: 807
dev.igc.0.iflib.txq1.txq_pidx: 851
dev.igc.0.iflib.txq1.no_tx_dma_setup: 0
dev.igc.0.iflib.txq1.txd_encap_efbig: 0
dev.igc.0.iflib.txq1.tx_map_failed: 0
dev.igc.0.iflib.txq1.no_desc_avail: 0
dev.igc.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq1.m_pullups: 0
dev.igc.0.iflib.txq1.mbuf_defrag: 0
dev.igc.0.iflib.txq1.cpu: 1
dev.igc.0.iflib.txq0.r_abdications: 0
dev.igc.0.iflib.txq0.r_restarts: 0
dev.igc.0.iflib.txq0.r_stalls: 0
dev.igc.0.iflib.txq0.r_starts: 701758
dev.igc.0.iflib.txq0.r_drops: 0
dev.igc.0.iflib.txq0.r_enqueues: 701912
dev.igc.0.iflib.txq0.ring_state: pidx_head: 1496 pidx_tail: 1496 cidx: 1496 state: IDLE
dev.igc.0.iflib.txq0.txq_cleaned: 1425785
dev.igc.0.iflib.txq0.txq_processed: 1425825
dev.igc.0.iflib.txq0.txq_in_use: 41
dev.igc.0.iflib.txq0.txq_cidx_processed: 417
dev.igc.0.iflib.txq0.txq_cidx: 377
dev.igc.0.iflib.txq0.txq_pidx: 418
dev.igc.0.iflib.txq0.no_tx_dma_setup: 0
dev.igc.0.iflib.txq0.txd_encap_efbig: 0
dev.igc.0.iflib.txq0.tx_map_failed: 0
dev.igc.0.iflib.txq0.no_desc_avail: 0
dev.igc.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igc.0.iflib.txq0.m_pullups: 0
dev.igc.0.iflib.txq0.mbuf_defrag: 0
dev.igc.0.iflib.txq0.cpu: 0
dev.igc.0.iflib.override_nrxds: 0
dev.igc.0.iflib.override_ntxds: 0
dev.igc.0.iflib.allocated_msix_vectors: 5
dev.igc.0.iflib.use_extra_msix_vectors: 0
dev.igc.0.iflib.use_logical_cores: 0
dev.igc.0.iflib.separate_txrx: 0
dev.igc.0.iflib.core_offset: 0
dev.igc.0.iflib.tx_abdicate: 0
dev.igc.0.iflib.rx_budget: 0
dev.igc.0.iflib.disable_msix: 0
dev.igc.0.iflib.override_qs_enable: 0
dev.igc.0.iflib.override_nrxqs: 0
dev.igc.0.iflib.override_ntxqs: 0
dev.igc.0.iflib.driver_version: 1
dev.igc.0.%iommu: rid=0x8400
dev.igc.0.%parent: pci7
dev.igc.0.%pnpinfo: vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:132:0:0 handle=\_SB_.PC02.RP04.PXSX
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) Ethernet Controller I226-V
dev.igc.%parent:
Comment 65 Kevin Bowling freebsd_committer freebsd_triage 2026-08-07 01:02:02 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #64)
I think the sysctl dump is post recovery?  The only clue is missed_packets, maybe indicating an RX stall.  It would help to capture two snapshots a few seconds apart before resetting while another host sends traffic: vmstat -i | grep igc, netstat -I igc0 -bd, and the sysctl dumps.

Please also run these before, during, and after:

pciconf -lc igc0

and the following script:

/bin/sh <<'EOF'
cap=$(pciconf -lc igc0 |
    sed -n 's/.*ecap 001e\[\([0-9A-Fa-f][0-9A-Fa-f]*\)\].*/\1/p')

if [ -z "$cap" ]; then
    echo "L1PM capability 001e not found"
    exit 1
fi

off=$((0x${cap} + 8))
set -- $(pciconf -r igc0 "$off")
ctl1=$1

printf 'L1PM capability offset: 0x%s\n' "$cap"
printf 'L1PM CTL1 offset:      0x%x\n' "$off"
printf 'L1PM CTL1 value:       0x%s\n' "$ctl1"

if [ $((0x${ctl1} & 0x4)) -eq 0 ]; then
    echo 'I226 ASPM L1.2 enable: CLEAR (workaround active)'
else
    echo 'I226 ASPM L1.2 enable: SET (workaround absent/lost)'
fi
EOF
Comment 66 Tomasz "CeDeROM" CEDRO 2026-08-07 01:25:37 UTC
That sysctl dump was from within the stall, interface was up but the connectivity failed, had to restart network by hand to get it running again (service netif stop; /etc/netstart).

Stuff below is after igc0 restart, got back up, but then I switched to ix0 and put igc0 down, no reboot so maybe something interesting is still here. I am working on ix0 right now. igc is extremely unreliable here and always craps itself in the middle or work.. how this is frustrating on your own local desktop now imagine you loose network on a remote machine.

#
# PCICONF
#

igc0@pci0:132:0:0:      class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
                 max read 512
                 link x1(x1) speed 5.0(5.0) ASPM L1(L1)
    ecap 0001[100] = AER 2 0 fatal 1 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 [XXX]
    ecap 0018[1c0] = LTR 1
    ecap 001f[1f0] = Precision Time Measurement 1
    ecap 001e[1e0] = L1 PM Substates 1

#
# VMSTAT
#

irq151: igc0:rxq0                1075529         61
irq152: igc0:rxq1                1905162        108
irq153: igc0:rxq2                 683268         39
irq154: igc0:rxq3                 630179         36
irq155: igc0:aq                        6          0

#
# NETSTAT
#

Name    Mtu Network   Address              Ipkts Ierrs Idrop       Ibytes    Opkts Oerrs       Obytes  Coll  Drop
igc0   1500 <Link#1>  [XXX]  8503350   322     0  11337485568  2336649     0    292561991     0     0

#
# FIX CHECK
#

L1PM capability offset: 0x1e0
L1PM CTL1 offset:      0x1e8
L1PM CTL1 value:       0x408c001b
I226 ASPM L1.2 enable: CLEAR (workaround active)
Comment 67 Kevin Bowling freebsd_committer freebsd_triage 2026-08-07 02:03:03 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #66)
I'm suspicious of vbox here, rx q0 is stalled and there isn't any strong hw lead left.  Does it happen without vbox?

/bin/sh <<'EOF'
read4()
{
    reg=$1
    name=$2
    idx=$((reg / 4))
    val=$(pciconf -D igc0 0x10 "$idx" 1 |
        hexdump -v -e '1/4 "%08x"')
    printf '%-5s [0x%04x] = 0x%s\n' "$name" "$reg" "$val"
}

read4 $((0x1514)) GPIE
read4 $((0x1524)) EIMS
read4 $((0x152c)) EIAC
read4 $((0x1530)) EIAM
EOF

upload these:
procstat -kk 0 > /tmp/igc-stall.kstack
procstat -t 0 > /tmp/igc-stall.threads
vmstat -i | grep igc
sleep 2
vmstat -i | grep igc
Comment 68 Tomasz "CeDeROM" CEDRO 2026-08-07 11:00:04 UTC
Yes IGC also hangs when vbox is not in use.. but I suspect it may speed things up.. maybe some driver from within the vm manipulates the nic.. on the other hand it is set to nat mode not bridge etc so it just should use generic tcp/ip transport.

Dumps attached in a moment (cannot paste here too long), this is from a fresh system right after power on, thanks :-)
Comment 69 Tomasz "CeDeROM" CEDRO 2026-08-07 11:04:54 UTC
Created attachment 273525 [details]
cederom igcdump 1786100514
Comment 70 Kevin Bowling freebsd_committer freebsd_triage 2026-08-07 11:18:22 UTC
Thanks, do the same logging again after a stall and add   pciconf -lce igc0 as well
Comment 71 Tomasz "CeDeROM" CEDRO 2026-08-07 11:25:51 UTC
pciconf -lce igc0 added :-) will send stall logs when it happens :-) thanks :-)
Comment 72 commit-hook freebsd_committer freebsd_triage 2026-08-08 00:39:29 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fd9a9cbcdf27e8dabec9c4ce3edf4c4f868f0cf2

commit fd9a9cbcdf27e8dabec9c4ce3edf4c4f868f0cf2
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-08-03 09:55:15 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-08-08 00:37:45 +0000

    igc: Disable PCIe L1.2 on I225

    I225 devices can incorrectly enter L1 substates while CLKREQ# is
    asserted, both while idle and in D3.  Disable ASPM and PCI-PM L1.2 on
    I225 to prevent the resulting packet loss.

    Keep the I226 workaround ASPM-only because it addresses a separate
    traffic exit latency observation.

    PR:             265714

    (cherry picked from commit 4a28d390f5fbae2483e88805559881b04ccf9a80)

 sys/dev/igc/if_igc.c   | 42 ++++++++++++++++++++++--------------------
 sys/dev/igc/igc_base.c | 29 +++++++++++++++++++++++++++++
 sys/dev/igc/igc_base.h |  1 +
 3 files changed, 52 insertions(+), 20 deletions(-)
Comment 73 commit-hook freebsd_committer freebsd_triage 2026-08-08 00:44:33 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3da5aba3f6131baaf76949d54ba6caa73fb055cd

commit 3da5aba3f6131baaf76949d54ba6caa73fb055cd
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-08-03 09:55:15 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-08-08 00:43:44 +0000

    igc: Disable PCIe L1.2 on I225

    I225 devices can incorrectly enter L1 substates while CLKREQ# is
    asserted, both while idle and in D3.  Disable ASPM and PCI-PM L1.2 on
    I225 to prevent the resulting packet loss.

    Keep the I226 workaround ASPM-only because it addresses a separate
    traffic exit latency observation.

    PR:             265714

    (cherry picked from commit 4a28d390f5fbae2483e88805559881b04ccf9a80)

 sys/dev/igc/if_igc.c   | 42 ++++++++++++++++++++++--------------------
 sys/dev/igc/igc_base.c | 29 +++++++++++++++++++++++++++++
 sys/dev/igc/igc_base.h |  1 +
 3 files changed, 52 insertions(+), 20 deletions(-)
Comment 74 Tomasz "CeDeROM" CEDRO 2026-08-08 17:05:25 UTC
Created attachment 273549 [details]
igc dump while and after network stall

cederom-igcdump-1786193471.log contains dump while in stall (no packets transport).
cederom-igcdump-1786193501.log contains dump after stall (network works okay).

network restart: service netif stop; sh /etc/netstart; service local_unbound restart
Comment 75 Kevin Bowling freebsd_committer freebsd_triage 2026-08-08 23:06:51 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #74)
That dump is helpful, we "caught" it insofar as EIMS=0x1e during the stall and 0x1f after restart so it lost a rearm somehow.  The L1.2 workaround is confirmed active.

There is an oddity on your particular system, in several of the pastes/dumps after the stall there is a PCIe AER "Unsupported Request".  I'm not sure what to make of that (it could be a platform or physical issue) but it is concerning.  You could also try disabling AER temporarily in the firmware to see if we are mishandling an event.

Next steps after another stall...

Manually kick q0 back into action:
pciconf -W igc0 0x10 0x1524 0x00000001

If that doesn't work then generate an interrupt:
pciconf -W igc0 0x10 0x1520 0x00000001

Run the read4 script after so we can see EIMS, does this recover it?  If so the issue is a lost interrupt.

Main e2aff50727cbe4cb5e99f825c2c6bd8a4915de67 is possibly relevant, more of a correctness thing but worth cherry-picking into your branch as I saw a similar issue with an I219 recently.
Comment 76 Tomasz "CeDeROM" CEDRO 2026-08-09 23:45:28 UTC
Created attachment 273586 [details]
igc stall 20260810-0140

Another stall dump provided :-)

These commands seem to have invalid syntax? Could not run:
pciconf -W igc0 0x10 0x1520 0x00000001
pciconf -W igc0 0x10 0x1524 0x00000001

Tried these below with no success in bringing back the network traffic:
pciconf -w igc0 0x1520 0x00000001
pciconf -w igc0 0x1524 0x00000001

I cannot find e2aff50727cbe4cb5e99f825c2c6bd8a4915de67 could you please provide url from cgit.freebsd.org or github?

I am working so far on the last branch here https://github.com/cederom/freebsd-src/commits/cederom-releng-14.4.8-igc_test_20260803 so the dump is from the same setup as last time.

Thank you :-)
Comment 77 Kevin Bowling freebsd_committer freebsd_triage 2026-08-10 02:34:11 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #76)
https://cgit.freebsd.org/src/commit/?id=e2aff50727cbe4cb5e99f825c2c6bd8a4915de67

It seems pciconf -W doesn't exist yet on 14.4 so that won't work (-w is different).
Comment 78 Tomasz "CeDeROM" CEDRO 2026-08-10 04:06:51 UTC
Got it thank you! :-)

% uname -a
FreeBSD hexagon 14.4-RELEASE-p8 FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260810-n273771-e99dffa6c4c8 GENERIC amd64

https://github.com/cederom/freebsd-src/commits/cederom-releng-14.4.8-igc_test_20260810/
Comment 79 Tomasz "CeDeROM" CEDRO 2026-08-11 20:28:24 UTC
Created attachment 273658 [details]
two igc stall dumps from 20260811.

Two more igc stalls with e2aff50727cbe4cb5e99f825c2c6bd8a4915de67 added does not seem to change anything :-(
Comment 80 Tomasz "CeDeROM" CEDRO 2026-08-11 23:51:19 UTC
I am switching now to IX (X540-T4) card. This IGC even if not completely dropping traffic seems unreliable anyways after some time drops traffic etc.

Please let me know if you need more tests Kevin, and Big Thank You for all of the fixes so far! :-)
Comment 81 Kevin Bowling freebsd_committer freebsd_triage 2026-08-12 21:44:42 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #80)
In your two new dumps the evidence is still RDH == RDT, that shouldn't happen during operation and it didn't only happen on queue 0 on the first stall.  One continuing oddity in all your dumps is PCIe/AER error so I would not rule out faulty HW (which could be the NIC or the mainboard) but they aren't indicating TLP errors or anything really conclusive.

You could try these to see if we catch it on the igc side:
https://cgit.freebsd.org/src/commit/?id=bbf93227fe9ee1f38b7db5477398c3564e5c84bc
https://cgit.freebsd.org/src/commit/?id=9f7633b932954162792e2caef4f6f0cd87e82f43

but I don't think the issue the first one covers would be that common.

Is the iommu active?

sysctl hw.iommu
sysctl dev.dmar
kenv | egrep 'iommu|dmar'
dmesg -a | egrep -i 'dmar|iommu|fault|remap|vbox|vmm'
kldstat
ps auxww | egrep -i 'vbox|virtualbox'
ifconfig -a
pciconf -t -v
Comment 82 Tomasz "CeDeROM" CEDRO 2026-08-12 22:30:37 UTC
Thank you Kevin :-)

The mainboard is new and still under warranty so I don't suspect a defect unless it was manufacturing defect or bad design. I took this one because it is one of few that still has some spare PCI-e slots and pretty decent chipset. I have reported Ethernet issues to the vendor some months ago and they sent this to HQ for verification but I got no response since then. This mainboard has thousands of options in BIOS [2] which most of them are set to Auto, I only choose these which I know about.. maybe this "AI"/"Auto" is the problem because mobo can change settings on each boot or even runtime no clue.

Ah I saw two days ago new BIOS showed up, will update and maybe that helps somehow :-)

Another Idea is to maybe search for this I226V on the PCIe and just try another one to exclude faulty hardware?

By the way there is one difference with IX driver - ifconfig shows "status: no carrier" when interface is down even if cable is attached, while IGC shows "status: active" when cable is attached but interface is down.



# uname -a
FreeBSD hexagon 14.4-RELEASE-p8 FreeBSD 14.4-RELEASE-p8 cederom-releng-14.4.8-igc_test_20260810-n273771-e99dffa6c4c8 GENERIC amd64

# sysctl hw.iommu
hw.iommu.batch_coalesce: 100
hw.iommu.tbl_pagecnt: 0
hw.iommu.dmar.timeout: 1000000

# sysctl dev.dmar
sysctl: unknown oid 'dev.dmar'

# kenv | egrep 'iommu|dmar'

# dmesg -a | egrep -i 'dmar|iommu|fault|remap|vbox|vmm'
[1] Trying to mount root from zfs:znvd/ROOT/default []...
[37] add net default: gateway XXX fib 0: route already in table
[38] ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled
[38] 00000 check-state :default
[38] 00000 allow tcp from me to any setup keep-state :default
[38] 00000 allow udp from me to any keep-state :default
[38] 00000 allow icmp from me to any keep-state :default
[38] 00000 allow ipv6-icmp from me to any keep-state :default
[45] Starting default mousedmoused: unable to open /dev/psm0: No such file or directory
[10978] vboxdrv: XXXXXXXXXXXXXXXX VMMR0.r0
[10978] vboxdrv: XXXXXXXXXXXXXXXX VBoxDDR0.r0
[10978] VMMR0InitVM: eflags=40246 fKernelFeatures=0x2 (SUPKERNELFEATURES_SMAP=1)
[25528] vboxdrv: XXXXXXXXXXXXXXXX VMMR0.r0
[25528] vboxdrv: XXXXXXXXXXXXXXXX VBoxDDR0.r0
[25528] VMMR0InitVM: eflags=40246 fKernelFeatures=0x2 (SUPKERNELFEATURES_SMAP=1)

# kldstat
Id Refs Address                Size Name
 1  140 0xffffffff80200000  1f44e80 kernel
 2    1 0xffffffff82145000     79b0 hidraw.ko
 3    1 0xffffffff8214d000     7808 cryptodev.ko
 4    1 0xffffffff82155000    133b0 ntb_hw_intel.ko
 5    2 0xffffffff82169000     6930 ntb.ko
 6    1 0xffffffff82170000     b758 cuse.ko
 7    1 0xffffffff8217c000     43c0 cpuctl.ko
 8    1 0xffffffff82181000     3870 hgame.ko
 9    5 0xffffffff82185000     4e80 hidmap.ko
10    1 0xffffffff8218a000    27670 fusefs.ko
11    3 0xffffffff821b2000    8e700 vboxdrv.ko
12    1 0xffffffff82242000    61b88 linux64.ko
13    7 0xffffffff822a4000    219a0 linux_common.ko
14    3 0xffffffff822c6000    6a688 linux.ko
15    1 0xffffffff82331000   5ee700 zfs.ko
16    1 0xffffffff82920000     5b18 sysctlinfo.ko
17    1 0xffffffff82926000     36c8 coretemp.ko
18    1 0xffffffff8292a000    26038 hwpmc.ko
19    1 0xffffffff82951000     3368 sysctlbyname_improved.ko
20    1 0xffffffff83b70000     3560 fdescfs.ko
21    1 0xffffffff83b74000   149458 nvidia-modeset.ko
22    1 0xffffffff83e00000  6791be0 nvidia.ko
23    1 0xffffffff83cbe000     3400 uchcom.ko
24    1 0xffffffff83cc2000     4e10 ucom.ko
25    1 0xffffffff83cc7000     2278 pty.ko
26    1 0xffffffff83cca000     73c0 linprocfs.ko
27    1 0xffffffff83cd2000     440c linsysfs.ko
28    1 0xffffffff83cd7000     3390 acpi_wmi.ko
29    1 0xffffffff83cdb000     21e8 hcons.ko
30    1 0xffffffff83cde000     21e8 hms.ko
31    1 0xffffffff83ce1000     21e8 hsctrl.ko
32    1 0xffffffff83ce4000     4250 ichsmb.ko
33    1 0xffffffff83ce9000     2178 smbus.ko
34    2 0xffffffff83cec000     4248 vboxnetflt.ko
35    2 0xffffffff83cf1000     abb8 netgraph.ko
36    1 0xffffffff83cfc000     31e0 ng_ether.ko
37    1 0xffffffff83d00000     55f0 vboxnetadp.ko
38    1 0xffffffff83d06000     3360 uhid.ko
39    1 0xffffffff83d0a000     3360 wmt.ko
40    1 0xffffffff83d0e000     f5b0 snd_uaudio.ko
41    1 0xffffffff83d1e000     4364 ums.ko
42    1 0xffffffff83d23000    28450 ipfw.ko
43    1 0xffffffff83d4c000     4b10 nullfs.ko

#  ps auxww | egrep -i 'vbox|virtualbox'
XXX    73148    0.0  0.3   584796  267312  -  SNs  23:04       0:04.30 /usr/local/lib/virtualbox/VirtualBox
XXX    74416    0.0  0.0    33356   16480  -  SN   23:04       0:04.98 /usr/local/lib/virtualbox/VBoxXPCOMIPCD
XXX    75694    0.0  0.0    96348   35212  -  SN   23:04       0:11.61 /usr/local/lib/virtualbox/VBoxSVC --auto-shutdown
root         887    0.0  0.0    13836    2316  1  S+   00:08       0:00.00 egrep -i vbox|virtualbox


#  ifconfig -a
igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	inet XXX netmask XXX broadcast XXX
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	media: Ethernet autoselect
	status: no carrier
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	media: Ethernet autoselect
	status: no carrier
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ix2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	media: Ethernet autoselect
	status: no carrier
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ix3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether XXX
	media: Ethernet autoselect
	status: no carrier
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet 127.0.0.1 netmask 0xff000000
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

#  pciconf -t -v
pciconf: illegal option -- t
usage: pciconf -l [-BbcevV] [device]
       pciconf -a device
       pciconf -r [-b | -h] device addr[:addr2]
       pciconf -w [-b | -h] device addr value
       pciconf -D [-b | -h | -x] device bar [start [count]]



Regarding the commits none of these two applies cleanly on my releng/14.4, below is the history that applies okay:

2f85437bf220 (HEAD -> cederom-releng-14.4.8-igc_test_20260813) igc: Remove invalid debug ring pointer iteration
0d212a99a907 igc: Add led(4) identification support
e7ec9d32123a igc: Avoid a signed shift while assembling the PHY ID
8580a90787de igc: Export EEE Low Power Idle counters
8556b22ae729 igc: Check PHY control register reads
1eaf086ad2ba igc: fix data type in MAC hash
b5b7f2eb2582 igc: fix data type in MAC hash
25a900eea8d4 igc: fix MAC address hash bit shift
7843123f8673 igc: fix iterator type
3d73fda88d7a igc: Avoid a signed multicast bitmap shift
e99dffa6c4c8 (cederom/cederom-releng-14.4.8-igc_test_20260810, cederom-releng-14.4.8-igc_test_20260810) igc: Correct descriptor control programming
18774c90fb02 (cederom/cederom-releng-14.4.8-igc_test_20260803, cederom-releng-14.4.8-igc_test_20260803) igc: Add VLAN hardware filtering
c1bc2da741d5 igc: Correct hardware error statistics
e80624657a23 igc: Work around I225 v1 minimum IPG erratum
8429f0391c11 igc: Disable PCIe L1.2 on I225
ea774409c9ca igc: Apply ASPM L1.2 workaround to all I226 devices
df561a9ea5f9 igc: Disable ASPM L1.2 on I226 to prevent RX stalls
4472d239ff5b igc(4): document adaptive interrupt moderation
53408cbd10c1 igc: count TSO wire segments in the AIM counters
f94e9349b389 igc: use packet-size AIM
fe2c8e215ccf igc: synchronize interrupt moderation state
fdf2a8ad9699 igc: make AIM counter sampling coherent
dc0fd5a3b87b igc: fix RX accounting for multi-descriptor packets
d6f358d9412c igc.4: Describe better
a3a185dba982 igc: use newly exposed RSS hash key API rather than ad-hoc hashing
518828a76cef pci: Add L1 PM definitions
d9b8b38e449f rss: Enable portions of RSS globally to enable symmetric hashing
3de2d29b088e (tag: release/14.4.0-p8, origin/releng/14.4, github/releng/14.4, cederom/releng/14.4, releng/14.4) Add UPDATING entries and bump version

I will take a look and sort out what else is missing. Hand edit merges but does not build :-P

Thank you :-)
Tomek


[1] https://www.asus.com/motherboards-components/motherboards/tuf-gaming/tuf-gaming-z890-pro-wifi/
[2] https://www.asus.com/motherboards-components/motherboards/tuf-gaming/tuf-gaming-z890-pro-wifi/helpdesk_manual?model2Name=TUF-GAMING-Z890-PRO-WIFI
Comment 83 Kevin Bowling freebsd_committer freebsd_triage 2026-08-13 15:50:15 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #82)
Do you know the exact model of the I225, is it an official intel card or an OEM one?  I'd like to try and buy one or have someone send me a broken one.
Comment 84 Tomasz "CeDeROM" CEDRO 2026-08-13 18:17:14 UTC
It is onboard NIC built into the TUF GAMING Z890-PRO WIFI mainboard. Otherwise I would replace already.

https://www.asus.com/motherboards-components/motherboards/tuf-gaming/tuf-gaming-z890-pro-wifi/

Backporting all stuff into 14.4 seems to be too hard for me sorry, there were too many changes in other places to track :-(
Comment 85 Tomasz "CeDeROM" CEDRO 2026-08-13 18:20:05 UTC
Kevin, I can see on Amazon I226V based pci-e cards are available for $15+ if you find one to play around I will refund (up to $50) so you have your own hardware to play with, please let me know :-)
Comment 86 Kevin Bowling freebsd_committer freebsd_triage 2026-08-13 18:43:36 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #85)
Thanks, I have I225 and I226 but I have never seen these issues.  The problem is specific to some cards and/or NVMs so I need to find an exact problem card.  Does anyone know if one of the amazon cards have this issue?
Comment 87 Tomasz "CeDeROM" CEDRO 2026-08-13 18:57:03 UTC
This may be as well caused by card firmware or the motherboard BIOS. The last BIOS I was using 3020 from April turned out Beta Version, the fresh one 3211 made my USB ports go from USB4/3 to 2 speeds (i.e. 160MB/s pendrive now reaches fascinating 40MB/s lol). Reverting to 3020 does not help. Also you have no idea what does "Auto" mean as there is no display which option is chosen by this Auto and if it changes anyhow on reboot / poweroff / runtime.

I have reported this USB issue to the vendor and they rapidly responded with "verify cables, update windows drivers, etc" heh.

Things get worse each time. Welcome to the Brave New World ;-)
Comment 88 karl 2026-08-13 20:08:21 UTC
I have a "microPC" that has two of these ports in it:

igc0: <Intel(R) Ethernet Controller I226-V> mem 0x80c00000-0x80cfffff,0x80d00000-0x80d03fff at device 0.0 on pci1
igc0: EEPROM V2.17-0 eTrack 0x80000303
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: c8:ff:bf:05:95:e2
igc0: netmap queues/slots: TX 4/1024, RX 4/1024
igc1: <Intel(R) Ethernet Controller I226-V> mem 0x80900000-0x809fffff,0x80a00000-0x80a03fff at device 0.0 on pci3
igc1: EEPROM V2.17-0 eTrack 0x80000303
igc1: Using 1024 TX descriptors and 1024 RX descriptors
igc1: Using 4 RX queues 4 TX queues
igc1: Using MSI-X interrupts with 5 vectors
igc1: Ethernet address: c8:ff:bf:05:95:e3
igc1: netmap queues/slots: TX 4/1024, RX 4/1024

This is a 15.1-STABLE build which I just refreshed today (so it has everything as of this morning) and I cannot reproduce hangs or other significant misbehavior with iperf3 running against a server with a Mellanox card on the other end through a 2.5Gbps connection (said Mellanox card is on 10G into the switch) with either single-stream or 4-stream parallel (e.g. -P 4)

This DID used to produce trouble some time back but with the current patches it does not and throughput is stable at 2.35Gbps -- for what its worth.  The machine itself is an N150 CPU Chineesium machine that used to be all over the place at very reasonable cost but the RAM insanity since the New Year has basically killed all of those off (prices have more than doubled and the "tiny box" ones have gone away entirely, which for this application is extremely annoying.)

CPU: Intel(R) N150 (806.40-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x239ca7eb<FSGSBASE,TSCADJ,BMI1,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,NFPUSG,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x98c007bc<UMIP,PKU,OSPKE,WAITPKG,GFNI,VAES,VPCLMULQDQ,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc184410<FSRM,MD_CLEAR,IBT,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  Structured Extended Features4=0x810
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x1580fd6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO,TAA_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 12884901888 (12288 MB)
avail memory = 12126695424 (11564 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >


The accumulated stats show LOTS of retries but no stalls or other misbehavior.

igc box as client:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-600.00 sec  54.8 GBytes   785 Mbits/sec  7447            sender
[  5]   0.00-600.02 sec  54.8 GBytes   785 Mbits/sec                  receiver
[  7]   0.00-600.00 sec  27.6 GBytes   395 Mbits/sec  10970            sender
[  7]   0.00-600.02 sec  27.6 GBytes   395 Mbits/sec                  receiver
[  9]   0.00-600.00 sec  27.2 GBytes   390 Mbits/sec  10970            sender
[  9]   0.00-600.02 sec  27.2 GBytes   390 Mbits/sec                  receiver
[ 11]   0.00-600.00 sec  54.8 GBytes   785 Mbits/sec  7446            sender
[ 11]   0.00-600.02 sec  54.8 GBytes   785 Mbits/sec                  receiver
[SUM]   0.00-600.00 sec   164 GBytes  2.35 Gbits/sec  36833             sender
[SUM]   0.00-600.02 sec   164 GBytes  2.35 Gbits/sec                  receiver

Same box as server (client is said Mellanox-equipped box coming this way):

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-600.00 sec  42.0 GBytes   602 Mbits/sec  145070            sender
[  5]   0.00-600.00 sec  42.0 GBytes   602 Mbits/sec                  receiver
[  7]   0.00-600.00 sec  42.9 GBytes   614 Mbits/sec  168729            sender
[  7]   0.00-600.00 sec  42.9 GBytes   614 Mbits/sec                  receiver
[  9]   0.00-600.00 sec  42.5 GBytes   609 Mbits/sec  104962            sender
[  9]   0.00-600.00 sec  42.5 GBytes   609 Mbits/sec                  receiver
[ 11]   0.00-600.00 sec  36.7 GBytes   525 Mbits/sec  138302            sender
[ 11]   0.00-600.00 sec  36.7 GBytes   525 Mbits/sec                  receiver
[SUM]   0.00-600.00 sec   164 GBytes  2.35 Gbits/sec  557063             sender
[SUM]   0.00-600.00 sec   164 GBytes  2.35 Gbits/sec                  receiver

iperf Done.

FreeBSD 15.1-STABLE stable/15-n284864-6b25004d1d18 GENERIC

Note that I do not have flow control enabled on the switch and it shows zero errors after this set of tests on either the client or server.
Comment 89 commit-hook freebsd_committer freebsd_triage 2026-09-04 05:51:03 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f1bd97bc567b293fcbe53b5f6b5e1cbe0d059cb1

commit f1bd97bc567b293fcbe53b5f6b5e1cbe0d059cb1
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-09-04 05:31:54 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-09-04 05:50:28 +0000

    sysutils/intel-nvmupdate-i225-i226: New port

    PR:             265714

 sysutils/Makefile                                  |   1 +
 sysutils/intel-nvmupdate-i225-i226/Makefile (new)  |  76 +++++++++++++++
 sysutils/intel-nvmupdate-i225-i226/distinfo (new)  |   5 +
 .../files/README.FreeBSD (new)                     |  80 ++++++++++++++++
 .../files/nvmupdate-i225-i226.in (new)             |   4 +
 .../files/nvmupdate-i226-it-1mb.cfg (new)          |  14 +++
 .../files/nvmupdate-i226-it-2mb.cfg (new)          |  14 +++
 .../files/nvmupdate-i226-lm-1mb.cfg (new)          |  14 +++
 .../files/nvmupdate-i226-lm-2mb.cfg (new)          |  14 +++
 .../files/nvmupdate.cfg (new)                      | 102 +++++++++++++++++++++
 .../files/pkg-message.in (new)                     |  45 +++++++++
 .../files/source_manifest.txt (new)                |  52 +++++++++++
 .../files/version.txt (new)                        |   6 ++
 sysutils/intel-nvmupdate-i225-i226/pkg-descr (new) |  11 +++
 sysutils/intel-nvmupdate-i225-i226/pkg-plist (new) |  24 +++++
 15 files changed, 462 insertions(+)
Comment 90 commit-hook freebsd_committer freebsd_triage 2026-09-04 08:01:33 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=d84474b55361078c2b79a438a466a10eea9da6a9

commit d84474b55361078c2b79a438a466a10eea9da6a9
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-09-04 07:57:25 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-09-04 07:59:28 +0000

    sysutils/intel-nvmupdate-i225-i226: Add Intel I225 images

    PR:             265714

 sysutils/intel-nvmupdate-i225-i226/Makefile        | 17 +++++++++----
 sysutils/intel-nvmupdate-i225-i226/distinfo        |  8 ++++---
 .../intel-nvmupdate-i225-i226/files/README.FreeBSD | 25 ++++++++++++-------
 .../intel-nvmupdate-i225-i226/files/nvmupdate.cfg  | 28 ++++++++++++++++++++++
 .../intel-nvmupdate-i225-i226/files/pkg-message.in | 19 ++++++++-------
 .../files/source_manifest.txt                      | 26 +++++++++++++++++++-
 .../intel-nvmupdate-i225-i226/files/version.txt    |  3 ++-
 sysutils/intel-nvmupdate-i225-i226/pkg-descr       | 19 ++++++++-------
 sysutils/intel-nvmupdate-i225-i226/pkg-plist       |  3 +++
 9 files changed, 113 insertions(+), 35 deletions(-)