This problem appeared with 13.3-BETA1, and I could reproduce it with stable/13-n257197-0efd4b792290 GENERIC Reproduce: - create a vnet jail and a pair of ng_eiface - configure one of the eiface als vnet.interface - start the jail - install openvpn (2.6.8) - start the openvon with some config, and terminate it again - stop the jail Normally the vnet.interface should now be back on the host, but now it isnt. It is possible to move the interface back to the host manually, with ifconfig -vnet, before stopping the jail. In either case the jail will never fully terminate, and stay in 'dying' state.
May you provide more information? 1. ngctl 2. ifconfig in jail and host
(In reply to Alexander Fedorov from comment #1) host # ngctl l -l There are 3 total nodes: Name: ntele1u Type: eiface ID: 00000004 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- ether ntele1l eiface 00000008 ether Name: ntele1l Type: eiface ID: 00000008 Num hooks: 1 Local hook Peer name Peer type Peer ID Peer hook ---------- --------- --------- ------- --------- ether ntele1u eiface 00000004 ether Name: ngctl10456 Type: socket ID: 0000000c Num hooks: 0 host # ifconfig vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE> ether 06:1d:92:01:04:01 inet 192.168.**.** netmask 0xffffffe0 broadcast 192.168.**.** media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> vtnet1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE> ether 00:a0:98:19:3b:56 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ntele1u: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=28<VLAN_MTU,JUMBO_MTU> ether 06:1d:92:09:02:05 hwaddr 58:9c:fc:00:79:5d inet 192.168.99.17 netmask 0xfffffffc broadcast 192.168.99.19 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> jail# ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ntele1l: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=28<VLAN_MTU,JUMBO_MTU> ether 06:1d:92:09:02:06 hwaddr 58:9c:fc:10:ff:b2 inet 192.168.99.18 netmask 0xfffffffc broadcast 192.168.99.19 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> --------------------------------------------- I tried to set vnet.interface=vtnet1 and this also got lost.
Show kldstat
I got a step further: it has noting to do with openvpn. Neither with netgraph. Just do this: - service jail start - jexec cat /dev/tun42 ^C - jexec ifconfig tun42 destroy - service jail stop
(In reply to Alexander Fedorov from comment #3) # kldstat Id Refs Address Size Name 1 10 0xffffffff80200000 2142e30 kernel 2 1 0xffffffff82510000 39c0 ng_socket.ko 3 2 0xffffffff82514000 aac8 netgraph.ko 4 1 0xffffffff8251f000 2210 ng_eiface.ko 5 1 0xffffffff82522000 2a08 mac_ntpd.ko
I have a suitable workaround for now, adding the concerned tunX to cloned_interfaces.
The issue appears because of change e900c81ede851f52 which fixes PR 273418
If you destroy the vnet-scoped tunX interface with `ifconfig tunX destroy`, does the vnet go away?
(In reply to Konstantin Belousov from comment #8) I'm not sure I understand your question, but I think the correct answer is No. I always destroy my tun devices before terminating the jail, because openvpn runs in chroot and fails to do so. And I've learned that one has to be 100% tidy for vnet to work smoothly. I just looked what Your code precisely does here in my testsuite: ifconfig tun42 create runs tun_clone_create(), clone_create() returns 1. -> no problem. cat /dev/tun43 runs tunclone(), clone_create() returns 1; runs tun_clone_create(), clone_create() returns 0; dev_ref() sets dev->si_refcount to 3. -> jid is now indestructible.
So, what do we do with this? Shall we declare it the recommended workaround to explicitely create the tun devices before starting openvpn instances, or should we try and figure out what went wrong with the former bugfix? Sorry to interrupt, but I have a bit too many dangling issues currently...