I add some configuretion on rc.conf [root@nas1 /home]# cat /etc/rc.conf |egrep "vlan111|ix0" cloned_interfaces="vlan111 vlan780" #ifconfig_ix0="up -tso4 -tso6 -lro -vlanhwtso description -=INTERFACE-TO-HERMES=-" ifconfig_ix0="up -tso4 -tso6 -lro -vlanhwtso description -=INTERFACE-TO-HERMES=-" ifconfig_vlan111="inet 192.168.0.8 netmask 255.255.255.224 vlan 111 vlandev ix0 description -=NAS-WORLD=-" [root@nas1 /home]# then do some commands: ifconfig vlan111 create Feb 28 12:05:34 nas1 kernel: ix0: link state changed to DOWN Feb 28 12:05:34 nas1 kernel: vlan111: link state changed to DOWN Feb 28 12:05:35 nas1 ntpd[1049]: Listen normally on 8 vlan111 192.168.0.8:123 Feb 28 12:05:36 nas1 kernel: ix0: link state changed to UP Feb 28 12:05:36 nas1 kernel: vlan111: link state changed to UP after this physical interface is flapping when I doo: ifconfig vlan111 destroy physical interface also is flapping Feb 28 12:05:57 nas1 kernel: ix0: link state changed to DOWN Feb 28 12:05:58 nas1 ntpd[1049]: Deleting interface #8 vlan111, 192.168.0.8#123, interface stats: received=0, sent=0, dropped=0, active_time=23 secs Feb 28 12:06:00 nas1 kernel: ix0: link state changed to UP
FreeBSD nas1 12.0-STABLE FreeBSD 12.0-STABLE r344575 NAS amd64
Does it help if you do "ifconfig ix0 -vlanhwtag -vlanhwfilter" before creating vlan? Some NICs chips re-establish link when host reprogramms their in-chip vlan mask. If you disable hardware-assisted vlan filtering to perform it at OS driver level, you shoud stop reprogramming chip while creating/destroying vlans.
Yes it helped.
Just a month ago, this did not happen.
(In reply to Alexey from comment #4) What revision do you run as per "uname -a" ?
Sorry, missed that you already mentioned r344575. Can you add revision you used "month ago" that had not this problem?
I have retest on: FreeBSD nas1 12.0-STABLE FreeBSD 12.0-STABLE r343868 NAS amd64 and this behavior is also present on this revision. This is a other server, with the same hardware configuration. There is no such problem on it. FreeBSD nas2 12.0-STABLE FreeBSD 12.0-STABLE r341831 NAS amd64 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: -=INTERFACE-TO-CORE2-SWITCH=- options=e000bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:e0:ed:2e:14:e1 media: Ethernet autoselect (10Gbase-Twinax <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
It seems, the code of ixgbe(4) driver got not significant changes in stable/12 between revisions 341831 and 343868 but iflib did. CC'ing marius@ who might have a glue. Alexey, please show part of "pciconf -lvvv" output for your ix0 device.
if_vlan(4) driver has changed in stable/12 between revisions 341831 and 343868, so CC'ing markj@ who changed it to give him a chance to look at this.
ix0@pci0:2:0:0: class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' class = network subclass = ethernet ix1@pci0:2:0:1: class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' class = network subclass = ethernet
JFYI: https://lists.freebsd.org/pipermail/freebsd-net/2018-November/052184.html
I rolled back on the current server to revision r341831 and it did not help, then I remembered that on the second server I used the drivers installed from the ports (/usr/ports/net/intel-ix-kmod/). Now when using these drivers - there are no problems and the physical interface does not flap. FreeBSD nas1 12.0-STABLE FreeBSD 12.0-STABLE r341831 NAS amd64 Mar 1 13:38:29 nas1 ntpd[1127]: Listen normally on 8 vlan111 192.168.0.8:123 Mar 1 13:38:32 nas1 ntpd[1127]: Deleting interface #8 vlan111, 192.168.0.8#123, interface stats: received=0, sent=0, dropped=0, active_time=3 secs [root@nas1 /home/pautina]# cat /etc/rc.conf |egrep "vlan111|ix1" cloned_interfaces="vlan111 vlan780" ifconfig_ix1="up -tso4 -tso6 -lro -vlanhwtso description -=INTERFACE-TO-APHRODITE=" ifconfig_vlan111="inet 192.168.0.8 netmask 255.255.255.224 vlan 111 vlandev ix1 description -=NAS-WORLD=-" [root@nas1 /home/pautina]# [root@nas1 /home/pautina]# ifconfig vlan111 vlan111: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: -=NAS-WORLD=- options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> ether e4:11:5b:9b:72:b4 inet 192.168.0.8 netmask 0xffffffe0 broadcast 192.168.0.31 groups: vlan vlan: 111 vlanpcp: 0 parent interface: ix1 media: Ethernet autoselect (10Gbase-Twinax <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> [root@nas1 /home/pautina]# [root@nas1 /home/pautina]# ifconfig ix1 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: -=INTERFACE-TO-APHRODITE= options=e000bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,RXCSUM_IPV6,TXCSUM_IPV6> ether e4:11:5b:9b:72:b4 media: Ethernet autoselect (10Gbase-Twinax <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> [root@nas1 /home/pautina]# [root@nas1 /home/pautina]# grep if_ix_updated /boot/loader.conf if_ix_updated_load="YES" [root@nas1 /home/pautina]# [root@nas1 /home/pautina]# kldstat Id Refs Address Size Name 1 9 0xffffffff80200000 12f04e0 kernel 2 1 0xffffffff814f1000 562f0 if_ix_updated.ko 3 1 0xffffffff81621000 acf mac_ntpd.ko [root@nas1 /home/pautina]# [root@nas1 /home/pautina]# dmesg |grep ix1 ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.3.6> port 0xe000-0xe01f mem 0xdf980000-0xdf9fffff,0xdfb00000-0xdfb03fff irq 17 at device 0.1 on pci2 ix1: Using MSI-X interrupts with 5 vectors ix1: Ethernet address: e4:11:5b:9b:72:b4 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: link state changed to UP [root@nas1 /home/pautina]# ix0@pci0:2:0:0: class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' class = network subclass = ethernet ix1@pci0:2:0:1: class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' class = network subclass = ethernet