Environment: amd64 on Xeon board. Long running 11.3Stable updated to 12Stable ~# uname -a FreeBSD hathor 12.0-STABLE FreeBSD 12.0-STABLE #0 r351834M: Sat Sep 7 17:20:37 AEST 2019 root@hathor:/usr/obj/prod/120101/D/K8/smallblocks/src/amd64.amd64/sys/hqdev-amd64-smp-vga amd64 Prior to the upgrade to 12Stable we built and tested 11.3Stable the day prior, everything was ok; only a few additions to the kernel config were applied (eg MAC_NTPD), otherwise unchanged. Summary: System is regularly updated from svnlite source (every 4-6 weeks) throughout the 11. series. The system was updated to 12Stable over the weekend. The system comprises multiple jailed environments including two i386 and a couple of amd64 jails. The i386 jails have a common base. Everything was working, until, the upgrade: Start jails - ok. Stop jails - ok. Restart jails - not ok. Further testing all (i386 and amd64)'s are caught in a dying state. Host reboot required. Details The test machine is quite dated and we used exec.poststop="umount -aF /etc/fstab.$name; umount $path/dev"; which, subsequent to reading Jamie's r349180 (that test is later) the problem remains We'll stop and start the jail, the jail.conf was slightly modified between tests. # jail -vr b3 b3: run command in jail: /bin/sh /etc/rc.shutdown b3: sent SIGTERM to: 1026 b3: removed b3: run command: /bin/sh -c umount -aF /etc/fstab.b3; umount /usr/jails/b3/dev jail: b3: mount.devfs: /usr/jails/b3/dev: not a mount point jail: b3: mount.fstab: /usr/jails/b3/pd/pd1: not a mount point jail: b3: mount.fstab: /usr/jails/b3/var/squid: not a mount point jail: b3: mount.fstab: /usr/jails/b3/var/mailctl: not a mount point jail: b3: mount.fstab: /usr/jails/b3/var/maildir: not a mount point jail: b3: mount.fstab: /usr/jails/b3/fifo: not a mount point jail: b3: mount.fstab: /usr/jails/b3/tmp: not a mount point jail: b3: mount.fstab: /usr/jails/b3/usr/ports: not a mount point jail: b3: mount.fstab: /usr/jails/b3/distfiles: not a mount point jail: b3: mount.fstab: /usr/jails/b3/packages: not a mount point jail: b3: mount.fstab: /usr/jails/b3/bj: not a mount point Rerun of jail removal after deleting the following historical workaround from jail.conf exec.poststop="umount -aF /etc/fstab.$name; umount $path/dev"; ~# jail -vr b3 b3: run command in jail: /bin/sh /etc/rc.shutdown b3: sent SIGTERM to: 26278 b3: removed b3: run command: /sbin/umount /usr/jails/b3/dev b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/pd/pd1 b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/squid b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/mailctl b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/maildir b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/fifo b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/tmp b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/usr/ports b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/distfiles b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/packages b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/bj Now to (attempt to) restart the jail # jail -vc b3 b3: run command: /sbin/mount -t nullfs -o rw /usr/jails/bj1 /usr/jails/b3/bj b3: run command: /sbin/mount -t nullfs -o ro /pd9/packages /usr/jails/b3/packages b3: run command: /sbin/mount -t nullfs -o ro /pd9/usr/distfiles /usr/jails/b3/distfiles b3: run command: /sbin/mount -t nullfs -o ro /usr/ports /usr/jails/b3/usr/ports b3: run command: /sbin/mount -t nullfs -o rw,noatime /tmp/b3 /usr/jails/b3/tmp b3: run command: /sbin/mount -t nullfs -o rw,noatime /m/b3/fifo /usr/jails/b3/fifo b3: run command: /sbin/mount -t nullfs -o rw,noatime /pd6/mail/maildir /usr/jails/b3/var/maildir b3: run command: /sbin/mount -t nullfs -o rw /pd6/mail/mailctl /usr/jails/b3/var/mailctl b3: run command: /sbin/mount -t nullfs -o rw,noatime /pd6/squid /usr/jails/b3/var/squid b3: run command: /sbin/mount -t nullfs -o rw,noatime /pd1 /usr/jails/b3/pd/pd1 b3: run command: /sbin/mount -t devfs -oruleset=6 . /usr/jails/b3/dev b3: jail_set(JAIL_CREATE) name=b3 host.hostname=b3.hs path=/usr/jails/b3 children.max=0 securelevel=3 enforce_statfs=1 jid=3 persist ip4.addr=10.0.7.91,10.0.5.91,127.0.5.91 devfs_ruleset=6 sysvmsg=new sysvsem=new sysvshm=new allow.raw_sockets allow.chflags host.hostuuid=00000000-0000-0000-0003-000000000003 host.hostid=3: File exists jail: b3: jail 3 already exists b3: run command: /sbin/umount /usr/jails/b3/dev b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/pd/pd1 b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/squid b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/mailctl b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/var/maildir b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/fifo b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/tmp b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/usr/ports b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/distfiles b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/packages b3: run command: /sbin/umount -ft nullfs /usr/jails/b3/bj ~# Can anyone reproduce by upgrading from 11Stable to 12Stable from source?
I'm experiencing something similar on 13.0-RELEASE: My jails are stuck at the dying stage, and will always show up in `jls -d`. Does not require rebooting in my case though.