Bug 230858 - Vlan Interface on ix1 doesn't pass any traffic on 12.0-ALPHA2 r338240
Summary: Vlan Interface on ix1 doesn't pass any traffic on 12.0-ALPHA2 r338240
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-24 06:12 UTC by kiwi
Modified: 2020-04-02 12:35 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description kiwi 2018-08-24 06:12:33 UTC
Hello,

As reported on twitter on FreeBSD 12.0-ALPHA2 r338240 on Supermicro X11DPU with 2 10G interfaces traffic going thru ix1 using VLAN interface doesn't work completly.

Have to set the ix1 in native mode to make this work.

Dmesg (not on r338240, bug still present) :
---<<BOOT>>---
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-ALPHA2 #0 r338182: Wed Aug 22 09:59:11 CEST 2018
    root@rsv-filer:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
WARNING: WITNESS option enabled, expect reduced performance.
VT(efifb): resolution 1024x768
CPU: Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz (2100.05-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x50654  Family=0x6  Model=0x55  Stepping=4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0xd39ffffb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,MPX,PQE,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,AVX512CD,AVX512BW,AVX512VL>
  Structured Extended Features2=0x8<PKU>
  Structured Extended Features3=0xc000000<IBPB,STIBP>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 68719476736 (65536 MB)
avail memory = 66447601664 (63369 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 hardware threads
random: unblocking device.
ioapic6: Changing APIC ID to 16
ioapic7: Changing APIC ID to 17
ioapic8: Changing APIC ID to 18
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-31 on motherboard
ioapic2 <Version 2.0> irqs 32-39 on motherboard
ioapic3 <Version 2.0> irqs 40-47 on motherboard
ioapic4 <Version 2.0> irqs 48-55 on motherboard
ioapic5 <Version 2.0> irqs 72-79 on motherboard
ioapic6 <Version 2.0> irqs 80-87 on motherboard
ioapic7 <Version 2.0> irqs 88-95 on motherboard
ioapic8 <Version 2.0> irqs 96-103 on motherboard
Launching APs: 1 26 6 8 4 20 10 2 12 16 14 24 22 18 9 15 31 29 30 27 3 28 25 7 5 17 11 21 13 23 19
Timecounter "TSC" frequency 2100054984 Hz quality 1000
random: entropy device external interface
[ath_hal] loaded
module_register_init: MOD_LOAD (vesa, 0xffffffff811011c0, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
kbd1 at kbdmux0
netmap: loaded module
nexus0
efirtc0: <EFI Realtime Clock> on motherboard
efirtc0: registered as a time-of-day clock, resolution 1.000000s
cryptosoft0: <software crypto> on motherboard
acpi0: <SUPERM SUPERM> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 350
Event timer "HPET1" frequency 24000000 Hz quality 340
Event timer "HPET2" frequency 24000000 Hz quality 340
Event timer "HPET3" frequency 24000000 Hz quality 340
Event timer "HPET4" frequency 24000000 Hz quality 340
Event timer "HPET5" frequency 24000000 Hz quality 340
Event timer "HPET6" frequency 24000000 Hz quality 340
Event timer "HPET7" frequency 24000000 Hz quality 340
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x508-0x50b on acpi0
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <dasp, performance counters> at device 8.1 (no driver attached)
pci0: <unknown> at device 17.0 (no driver attached)
pci0: <unknown> at device 17.1 (no driver attached)
ahci0: <Intel Lewisburg AHCI SATA controller> port 0x3070-0x3077,0x3060-0x3063,0x3020-0x303f mem 0x9d206000-0x9d207fff,0x9d209000-0x9d2090ff,0x9d180000-0x9d1fffff irq 16 at device 17.5 on pci0
ahci0: AHCI v1.31 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
xhci0: <Intel Lewisburg USB 3.0 controller> mem 0x383ffff00000-0x383ffff0ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0: waiting for BIOS to give up control
xhci_interrupt: host controller halted
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
pci0: <simple comms> at device 22.4 (no driver attached)
ahci1: <Intel Lewisburg AHCI SATA controller> port 0x3050-0x3057,0x3040-0x3043,0x3000-0x301f mem 0x9d204000-0x9d205fff,0x9d208000-0x9d2080ff,0x9d100000-0x9d17ffff irq 16 at device 23.0 on pci0
ahci1: AHCI v1.31 with 8 6Gbps ports, Port Multiplier not supported
ahcich6: <AHCI channel> at channel 0 on ahci1
ahcich7: <AHCI channel> at channel 1 on ahci1
ahcich8: <AHCI channel> at channel 2 on ahci1
ahcich9: <AHCI channel> at channel 3 on ahci1
ahcich10: <AHCI channel> at channel 4 on ahci1
ahcich11: <AHCI channel> at channel 5 on ahci1
ahcich12: <AHCI channel> at channel 6 on ahci1
ahcich13: <AHCI channel> at channel 7 on ahci1
ahciem1: <AHCI enclosure management bridge> on ahci1
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
vgapci0: <VGA-compatible display> port 0x2000-0x207f mem 0x9c000000-0x9cffffff,0x9d000000-0x9d01ffff irq 17 at device 0.0 on pci3
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
pci0: <serial bus> at device 31.5 (no driver attached)
pcib4: <ACPI Host-PCI bridge> on acpi0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 39 at device 0.0 on pci4
pci5: <ACPI PCI bus> on pcib5
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver> port 0x5020-0x503f mem 0xaab80000-0xaabfffff,0xaac04000-0xaac07fff irq 32 at device 0.0 on pci5
ix0: using 2048 tx descriptors and 2048 rx descriptors
ix0: msix_init qsets capped at 16
ix0: pxm cpus: 16 queue msgs: 63 admincnt: 1
ix0: using 16 rx queues 16 tx queues 
ix0: Using MSIX interrupts with 17 vectors
ix0: allocated for 16 queues
ix0: allocated for 16 rx queues
ix0: Ethernet address: 0c:c4:7a:f4:34:a0
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 16/2048, RX 16/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver> port 0x5000-0x501f mem 0xaab00000-0xaab7ffff,0xaac00000-0xaac03fff irq 36 at device 0.1 on pci5
ix1: using 2048 tx descriptors and 2048 rx descriptors
ix1: msix_init qsets capped at 16
ix1: pxm cpus: 16 queue msgs: 63 admincnt: 1
ix1: using 16 rx queues 16 tx queues 
ix1: Using MSIX interrupts with 17 vectors
ix1: allocated for 16 queues
ix1: allocated for 16 rx queues
ix1: Ethernet address: 0c:c4:7a:f4:34:a1
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 16/2048, RX 16/2048
pcib6: <ACPI PCI-PCI bridge> irq 39 at device 2.0 on pci4
pci6: <ACPI PCI bus> on pcib6
nvme0: <Samsung PM1725a> mem 0xaae10000-0xaae13fff irq 39 at device 0.0 on pci6
pcib7: <ACPI PCI-PCI bridge> irq 39 at device 3.0 on pci4
pci7: <ACPI PCI bus> on pcib7
nvme1: <Samsung PM1725a> mem 0xaad10000-0xaad13fff irq 39 at device 0.0 on pci7
pcib8: <ACPI Host-PCI bridge> on acpi0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> irq 47 at device 0.0 on pci8
pci9: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> irq 47 at device 1.0 on pci8
pci10: <ACPI PCI bus> on pcib10
pcib11: <ACPI Host-PCI bridge> on acpi0
pci11: <ACPI PCI bus> on pcib11
pcib12: <ACPI PCI-PCI bridge> irq 55 at device 2.0 on pci11
pci12: <ACPI PCI bus> on pcib12
mpr0: <Avago Technologies (LSI) SAS3008> port 0x9000-0x90ff mem 0xc5e40000-0xc5e4ffff,0xc5e00000-0xc5e3ffff irq 55 at device 0.0 on pci12
mpr0: Firmware: 15.00.03.00, Driver: 18.03.00.00-fbsd
mpr0: IOCCapabilities: 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastPath,RDPQArray>
pci11: <dasp, performance counters> at device 14.0 (no driver attached)
pci11: <dasp, performance counters> at device 15.0 (no driver attached)
pci11: <dasp, performance counters> at device 18.0 (no driver attached)
pci11: <dasp, performance counters> at device 18.1 (no driver attached)
pcib13: <ACPI Host-PCI bridge> on acpi0
pci13: <ACPI PCI bus> on pcib13
pci13: <dasp, performance counters> at device 8.1 (no driver attached)
pcib14: <ACPI Host-PCI bridge> on acpi0
pci14: <ACPI PCI bus> on pcib14
pcib15: <ACPI PCI-PCI bridge> irq 87 at device 2.0 on pci14
pci15: <ACPI PCI bus> on pcib15
pcib16: <ACPI PCI-PCI bridge> irq 87 at device 3.0 on pci14
pci16: <ACPI PCI bus> on pcib16
pcib17: <ACPI Host-PCI bridge> on acpi0
pci17: <ACPI PCI bus> on pcib17
pcib18: <ACPI Host-PCI bridge> on acpi0
pci18: <ACPI PCI bus> on pcib18
pci18: <dasp, performance counters> at device 14.0 (no driver attached)
pci18: <dasp, performance counters> at device 15.0 (no driver attached)
pci18: <dasp, performance counters> at device 18.0 (no driver attached)
pci18: <dasp, performance counters> at device 18.1 (no driver attached)
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xce000-0xcefff pnpid ORM0000 on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ugen0.1: <0x8086 XHCI root HUB> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
nvd0: <SAMSUNG MZWLL800HEHP-00003> NVMe namespace
nvd0: 763097MB (1562824368 512 byte sectors)
nvd1: <SAMSUNG MZWLL800HEHP-00003> NVMe namespace
nvd1: 763097MB (1562824368 512 byte sectors)
uhub0: 26 ports with 26 removable, self powered
mpr0: Found device <c01<SspTarg,Direct>,End Device> <6.0Gbps> handle<0x0009> enclosureHandle<0x0001> slot 0
mpr0: At enclosure level 0 and connector name (    )
lock order reversal:
mpr0:  1st 0xfffff802d0d254d0 CAM device lock (CAM device lock) @ /usr/src/sys/cam/cam_xpt.c:831
 2nd 0xfffffe00befa2218 MPR lock (MPR lock) @ /usr/src/sys/cam/cam_xpt.c:2808
stack backtrace:
#0 0xffffffff80c02553 at witnesFound device <c01<SspTarg,Direct>,End Device> <3.0Gbps> handle<0x000a> enclosureHandle<0x0001> slot 1
mpr0: At enclosure level 0 and connector name (    )
s_debugger+0x73
#1 0xffffffff80c023d4 at witness_checkorder+0xe34
#2 0xffffffff80b74c8c at __mtx_lock_flags+0x9c
#3 0xffffffugen0.2: <vendor 0x0557 product 0x7000> at usbus0
ff8034a24a at xpt_action_default+0x1ba
#4 0xffffffff8035f1e8 at scsi_scan_bus+0x68
mpr0: #5 0xffffffff80350e51 at xpt_scanner_thread+0x141
#6 0xffffffff80b55334 at fork_exit+0x84
#7 0xffffffff8105941e at fork_trampoline+0xe
Found device <c01<SspTarg,Direct>,End Device> <12.0Gbps> handle<0x000b> enclosureHandle<0x0001> slot 2
mpr0: At enclosure level 0 and connector name (    )
mpr0: uhub1Found device <c01<SspTarg,Direct>,End Device> <12.0Gbps> handle<0x000c> enclosureHandle<0x0001> slot 3
mpr0: At enclosure level 0 and connector name (    )
mpr0:  on uhub0
uhub1: Found device <c01<SspTarg,Direct>,End Device> <12.0Gbps> handle<0x000d> enclosureHandle<0x0001> slot 4
mpr0: At enclosure level 0 and connector name (    )
mpr0: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 1> on usbus0
Found device <c01<SspTarg,Direct>,End Device> <12.0Gbps> handle<0x000e> enclosureHandle<0x0001> slot 5
mpr0: At enclosure level 0 and connector name (    )
uhub1: 4 ports with 3 removable, self powered
ugen0.3: <vendor 0x0557 product 0x2419> at usbus0
ukbd0 on uhub1
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd2 at ukbd0
ugen0.4: <vendor 0x048d product 0x1172> at usbus0
umass0 on uhub0
umass0: <vendor 0x048d product 0x1172, class 0/0, rev 2.00/1.00, addr 3> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass0:17:0: Attached to scbus17
da2 at mpr0 bus 0 scbus16 target 4 lun 0
da2: <SEAGATE ST1000NX0453 N004> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number W4714PYH0000E8110EQE
da2: 1200.000MB/s transfers
da2: Command Queueing enabled
da2: 953869MB (1953525168 512 byte sectors)
da3 at mpr0 bus 0 scbus16 target 5 lun 0
da3: <SEAGATE ST1000NX0453 N004> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number W4714Q0R0000E810FKY7
da3: 1200.000MB/s transfers
da3: Command Queueing enabled
da3: 953869MB (1953525168 512 byte sectors)
da4 at mpr0 bus 0 scbus16 target 6 lun 0
da4: <SEAGATE ST1000NX0453 N004> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number W4714PYY0000E81158Q4
da4: 1200.000MB/s transfers
da4: Command Queueing enabled
da4: 953869MB (1953525168 512 byte sectors)
da5 at mpr0 bus 0 scbus16 target 7 lun 0
da5: <SEAGATE ST1000NX0453 N004> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number W4714QZ40000E8111C4T
da5: 1200.000MB/s transfers
da5: Command Queueing enabled
da5: 953869MB (1953525168 512 byte sectors)
da1 at mpr0 bus 0 scbus16 target 3 lun 0
da1: <HP DG072A9BB7 HPD0> Fixed Direct Access SPC-3 SCSI device
da1: Serial Number B365P710BTFJ0702
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 70007MB (143374738 512 byte sectors)
da6 at umass-sim0 bus 0 scbus17 target 0 lun 0
da6: <  0.00> Removable Direct Access SPC-2 SCSI device
da6: Serial Number B1311020000020
da6: 40.000MB/s transfers
da6: 3854MB (7892992 512 byte sectors)
da6: quirks=0x2<NO_6_BYTE>
da0 at mpr0 bus 0 scbus16 target 2 lun 0
da0: <HP EG0300FCSPH HPD0> Fixed Direct Access SPC-3 SCSI device
da0: Serial Number 14V0A0JHFTM91405
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 286102MB (585937500 512 byte sectors)
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ses1 at ahciem1 bus 0 scbus15 target 0 lun 0
ses1: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses1: SEMB SES Device
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from zfs:rsvfiler/ROOT/default []...
GEOM_ELI: Device da1p2.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: software
lo0: link state changed to UP
lock order reversal: (sleepable after non-sleepable)
 1st 0xfffff802e1167988 vlan (vlan) @ /usr/src/sys/net/if_vlan.c:1931
 2nd 0xfffff800059b0160 iflib ctx lock (iflib ctx lock) @ /usr/src/sys/net/iflib.c:4198
stack backtrace:
#0 0xffffffff80c02553 at witness_debugger+0x73
#1 0xffffffff80c023d4 at witness_checkorder+0xe34
#2 0xffffffff80b9fe48 at _sx_xlock+0x68
#3 0xffffffff80cbeeac at iflib_if_ioctl+0x8c
#4 0xffffffff80ca01da at if_addmulti+0x24a
#5 0xffffffff80cb11c6 at vlan_setmulti+0x196
#6 0xffffffff80cb0349 at vlan_ioctl+0xa9
#7 0xffffffff80ca01da at if_addmulti+0x24a
#8 0xffffffff80d26a9a at in_joingroup_locked+0x23a
#9 0xffffffff80d26832 at in_joingroup+0x42
#10 0xffffffff80d21b28 at in_control+0xb08
#11 0xffffffff80c9f1f1 at ifioctl+0x1861
#12 0xffffffff80c07d79 at kern_ioctl+0x2b9
#13 0xffffffff80c07a4e at sys_ioctl+0x15e
#14 0xffffffff8107ea21 at amd64_syscall+0x281
#15 0xffffffff81058d1d at fast_syscall_common+0x101
ix0: link state changed to UP
vlan3401: link state changed to UP
ix1: link state changed to UP
vlan3402: link state changed to UP
ioat0: <SKX IOAT> mem 0x383ffff2c000-0x383ffff2ffff irq 26 at device 4.0 on pci0
ioat0: Capabilities: f0006e1<DMA_with_DIF,PQ,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat1: <SKX IOAT> mem 0x383ffff28000-0x383ffff2bfff irq 27 at device 4.1 on pci0
ioat1: Capabilities: f0006e1<DMA_with_DIF,PQ,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat2: <SKX IOAT> mem 0x383ffff24000-0x383ffff27fff irq 26 at device 4.2 on pci0
ioat2: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat3: <SKX IOAT> mem 0x383ffff20000-0x383ffff23fff irq 27 at device 4.3 on pci0
ioat3: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat4: <SKX IOAT> mem 0x383ffff1c000-0x383ffff1ffff irq 26 at device 4.4 on pci0
ioat4: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat5: <SKX IOAT> mem 0x383ffff18000-0x383ffff1bfff irq 27 at device 4.5 on pci0
ioat5: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat6: <SKX IOAT> mem 0x383ffff14000-0x383ffff17fff irq 26 at device 4.6 on pci0
ioat6: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat7: <SKX IOAT> mem 0x383ffff10000-0x383ffff13fff irq 27 at device 4.7 on pci0
ioat7: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat8: <SKX IOAT> mem 0x393ffff1c000-0x393ffff1ffff irq 74 at device 4.0 on pci13
ioat8: Capabilities: f0006e1<DMA_with_DIF,PQ,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat9: <SKX IOAT> mem 0x393ffff18000-0x393ffff1bfff irq 75 at device 4.1 on pci13
ioat9: Capabilities: f0006e1<DMA_with_DIF,PQ,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat10: <SKX IOAT> mem 0x393ffff14000-0x393ffff17fff irq 74 at device 4.2 on pci13
ioat10: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat11: <SKX IOAT> mem 0x393ffff10000-0x393ffff13fff irq 75 at device 4.3 on pci13
ioat11: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat12: <SKX IOAT> mem 0x393ffff0c000-0x393ffff0ffff irq 74 at device 4.4 on pci13
ioat12: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat13: <SKX IOAT> mem 0x393ffff08000-0x393ffff0bfff irq 75 at device 4.5 on pci13
ioat13: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat14: <SKX IOAT> mem 0x393ffff04000-0x393ffff07fff irq 74 at device 4.6 on pci13
ioat14: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ioat15: <SKX IOAT> mem 0x393ffff00000-0x393ffff03fff irq 75 at device 4.7 on pci13
ioat15: Capabilities: f0004e1<DMA_with_DIF,Extended_APIC_ID,Block_Fill,Move_CRC,Page_Break>
ums0 on uhub1
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
ix1: promiscuous mode enabled
vlan3402: promiscuous mode enabled
lock order reversal:
 1st 0xffffffff81fc99e8 bpf global lock (bpf global lock) @ /usr/src/sys/net/bpf.c:792
 2nd 0xffffffff81fcac20 vlan_sx (vlan_sx) @ /usr/src/sys/net/if_vlan.c:1913
stack backtrace:
#0 0xffffffff80c02553 at witness_debugger+0x73
#1 0xffffffff80c023d4 at witness_checkorder+0xe34
#2 0xffffffff80b9fe48 at _sx_xlock+0x68
#3 0xffffffff80cb04eb at vlan_ioctl+0x24b
#4 0xffffffff80c9fca8 at if_setflag+0xf8
#5 0xffffffff80c9fb4c at ifpromisc+0x2c
#6 0xffffffff80c940c5 at bpf_detachd_locked+0x225
#7 0xffffffff80c96eeb at bpf_dtor+0x8b
#8 0xffffffff80a4c85e at devfs_destroy_cdevpriv+0x8e
#9 0xffffffff80a4ffc5 at devfs_close_f+0x65
#10 0xffffffff80b40a65 at closef+0x1f5
#11 0xffffffff80b3dae0 at closefp+0xa0
#12 0xffffffff8107ea21 at amd64_syscall+0x281
#13 0xffffffff81058d1d at fast_syscall_common+0x101
ix1: promiscuous mode disabled
vlan3402: promiscuous mode disabled
ix1: promiscuous mode enabled
lock order reversal:
 1st 0xffffffff81fc99e8 bpf global lock (bpf global lock) @ /usr/src/sys/net/bpf.c:792
 2nd 0xfffff800080be960 iflib ctx lock (iflib ctx lock) @ /usr/src/sys/net/iflib.c:4180
stack backtrace:
#0 0xffffffff80c02553 at witness_debugger+0x73
#1 0xffffffff80c023d4 at witness_checkorder+0xe34
#2 0xffffffff80b9fe48 at _sx_xlock+0x68
#3 0xffffffff80cbf10b at iflib_if_ioctl+0x2eb
#4 0xffffffff80c9fca8 at if_setflag+0xf8
#5 0xffffffff80c9fb4c at ifpromisc+0x2c
#6 0xffffffff80c940c5 at bpf_detachd_locked+0x225
#7 0xffffffff80c96eeb at bpf_dtor+0x8b
#8 0xffffffff80a4c85e at devfs_destroy_cdevpriv+0x8e
#9 0xffffffff80a4ffc5 at devfs_close_f+0x65
#10 0xffffffff80b40a65 at closef+0x1f5
#11 0xffffffff80b3dae0 at closefp+0xa0
#12 0xffffffff8107ea21 at amd64_syscall+0x281
#13 0xffffffff81058d1d at fast_syscall_common+0x101
ix1: promiscuous mode disabled
ix1: promiscuous mode enabled
vlan3402: promiscuous mode enabled
ix1: promiscuous mode disabled
vlan3402: promiscuous mode disabled
ix1: promiscuous mode enabled
vlan3402: promiscuous mode enabled
ix1: promiscuous mode disabled
vlan3402: promiscuous mode disabled
Comment 1 Harald Schmalzbauer 2018-08-24 14:27:29 UTC
Can you please check if reducing MTU to 1468 (on the if_vlan(4) child) has any influence?

If so, please cross-link https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230510

-harry
Comment 2 kiwi 2018-08-27 11:34:45 UTC
vlan3402: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1468
        options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:f4:34:a1
        inet 10.194.101.101 netmask 0xffffff00 broadcast 10.194.101.255 
        groups: vlan 
        vlan: 3402 vlanpcp: 0 parent interface: ix1
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Reducting the MTU doesn't seems to have any influence... 

Note that r338333 have same issue :(
Comment 3 Navdeep Parhar freebsd_committer 2018-08-27 16:00:09 UTC
> FreeBSD 12.0-ALPHA2 #0 r338182: Wed Aug 22 09:59:11 CEST 2018

VLAN tx was broken in r337943 and then fixed in r338305, so this particular
revision is definitely affected.  That bug was tracked via PR 230794.

But the submitter mentioned that r338333 doesn't work either so this may
not be a dup.
Comment 4 Piotr Pietruszewski 2018-09-05 15:26:06 UTC
Hello,

does this problem persist? If so, have you had a chance to check it on 12.0-ALPHA4?

What NIC are you using? Would you please show your configuration of parent interface (ix1)?

Is the machine connected to switch or do you use back-to-back configuration? If the former, would you please check if the problem occurs also with a back-to-back connection?

Peter
Comment 5 kiwi 2018-09-10 11:17:16 UTC
Hi, 

# uname -a
FreeBSD rsv-filer 12.0-ALPHA5 FreeBSD 12.0-ALPHA5 #0 r338557: Mon Sep 10 12:25:04 CEST 2018     root@rsv-filer:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG  amd64

Problem "seems" to have changed from pass no traffic to do not detect link on parent interface (I have this behavior on lagg on 11.2) :

ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:f4:34:a1
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan3402: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 0c:c4:7a:f4:34:a1
        inet 10.194.101.101 netmask 0xffffff00 broadcast 10.194.101.255 
        groups: vlan 
        vlan: 3402 vlanpcp: 0 parent interface: ix1
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Configuration is done using rc.conf :

ifconfig_ix1="up"
vlans_ix1="vlan3402"
create_args_vlan3402="vlan 3402"
ifconfig_vlan3402="10.194.101.101/24"

This machine is connected to a Cisco 6800 thru a Twinax cable, so the media should be : media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)

On the switch the configuration is pretty simple :
interface TenGigabitEthernet3/5
 description Server: rsv-filer
 switchport
 switchport trunk native vlan 3402
 switchport mode trunk
 switchport trunk allowed vlan 3401-3403
 switchport nonegotiate
 mtu 9216
 logging event link-status
 flowcontrol send on
 spanning-tree portfast edge trunk
 spanning-tree bpduguard enable
!
On the switch side the port is not shut because any security stuff...

Regards
Comment 6 kiwi 2018-09-10 11:25:49 UTC
Hum...

Addition: 
# kldstat
Id Refs Address                Size Name
 1   23 0xffffffff80200000  2450510 kernel
 2    1 0xffffffff82652000     22b0 cc_htcp.ko
 3    1 0xffffffff82655000   3a7ca8 zfs.ko
 4    2 0xffffffff829fd000     a4f0 opensolaris.ko
 5    1 0xffffffff82f12000     7d08 ioat.ko
 6    1 0xffffffff82f1a000     1800 uhid.ko
 7    1 0xffffffff82f1c000     23a8 ums.ko
 8    1 0xffffffff82f1f000      acf mac_ntpd.ko

Seems that if_vlan.ko is not loaded. 

Added it into loader.conf (and rebooted), link is up but issue is *still* there unfortunatly.
Comment 7 Piotr Pietruszewski 2018-09-12 13:37:54 UTC
Hello,

we have trouble reproducing this bug in our environment. Could you please provide more details?

1. What NIC is causing the problem?
2. Have you tried connecting it back-to-back? If so, were there any differences in behaviour you previously described?
3. Is there any additional configuration applied on this interface?
Comment 8 kiwi 2018-09-13 13:39:16 UTC
Hello 

1. Both nic make me problems. ix0 and ix1 (see the first dmesg).

I use usualy only ix1 to avoid time consuming issues to get back access to this servers (it is in a remote location, even with the IPMi, it can be a pain in the ass have access)

2. back-to-back is not possible to test, the servers is in a remote location :( and is "more or less" in production (shared storage dor esxi)

3. Only /etc/rc.conf configuration is done.

The motherboard is a X11DPU from supermicro (with latest BIOS used, which wad a pain to update as well).

Notice I get 4 servers like this (on 2 differents datacenter), 2 of them are currently working like a charm on ESX 6.7 (with vlan / mtu ....) the other two are used as storage on Freebsd.

These 2 freebsd servers was never being stable on FreeBSD 11.2 (nvme issues with zfs cache / log stuff) and network on freebsd 11.2 was a bit tricky (lagg that doesn't up ix1 for example).

Either this is an issue on FreeBSD with Supermicro X11DPU (maybe the hardware has a bug?) or something else ? Anyway on ESXi 6.7 (and 6.5) it working without any pain.

Rergards,
Xavier
Comment 9 Mark Linimon freebsd_committer freebsd_triage 2018-10-09 15:51:06 UTC
Is this a regression from an earlier install, or is this a new install?
Comment 10 kiwi 2020-04-02 12:35:59 UTC
Well... Since I didn't try for more than 2 years I close this bug... Seems to be fixed... somewhere.