I upgraded four systems last night. Three of them have problems with sshd after running freebsd-update on 9.3R. The i386 host did not start sshd: # service sshd start Performing sanity check on sshd configuration. /usr/sbin/sshd: Undefined symbol "ssh_explicit_bzero" /etc/rc.d/sshd: WARNING: failed precmd routine for sshd All jails on two amd64 hosts failed to start sshd. The fourth host (amd64) was not affected. There is no /usr/lib/private in any jail nor on the i386 host which failed to start. Both amd64 hosts have that directory. Erwin Lansing reports same problem see also: https://lists.freebsd.org/pipermail/freebsd-stable/2014-July/079334.html
Similar experience here. Simple "sudo freebsd-update fetch" "sudo freebsd-update install" on a amd64 VMware virtual machine, no jails involved, going from (probably) 9.3-p5 to 9.3-p8. No /usr/lib/private either.
I have a KVM for the busted i386 system if forensics will help.
The PR states the version is 9.2-STABLE. Please confirm what the FreeBSD OS version was prior to the upgrade.
Dan, if you can provide access, I would like to investigate.
Sorry, it was 9.3-RELEASE prior to running freebsd-update. It was most likely on 9.3-RELEASE-p5
Glen: I will find you on IRC.
FYI, the jails were running 9.2 based on what I find here: # mount -t zfs system/usr/jails.KEEP/basejail.OLD /mnt # cd /mnt/bin # file sh sh: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.2, stripped
From the i386 system: /usr/sbin/sshd: libssh.so.5 => /usr/lib/libssh.so.5 (0x280b2000) libutil.so.9 => /lib/libutil.so.9 (0x28106000) libwrap.so.6 => /usr/lib/libwrap.so.6 (0x28119000) libpam.so.5 => /usr/lib/libpam.so.5 (0x28121000) libbsm.so.3 => /usr/lib/libbsm.so.3 (0x2812c000) libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x28144000) libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x2815b000) libkrb5.so.10 => /usr/lib/libkrb5.so.10 (0x28164000) libhx509.so.10 => /usr/lib/libhx509.so.10 (0x281c4000) libasn1.so.10 => /usr/lib/libasn1.so.10 (0x281fa000) libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x28271000) libroken.so.10 => /usr/lib/libroken.so.10 (0x28273000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x28283000) libcrypto.so.6 => /lib/libcrypto.so.6 (0x282a8000) libz.so.6 => /lib/libz.so.6 (0x2840b000) libthr.so.3 => /lib/libthr.so.3 (0x2841f000) libc.so.7 => /lib/libc.so.7 (0x28440000) libmd.so.5 => /lib/libmd.so.5 (0x28572000)
From the i386 system: $ ls -l /usr/lib/libssh.so lrwxr-xr-x 1 root wheel 11 Feb 18 2011 /usr/lib/libssh.so -> libssh.so.5 -r--r--r-- 1 root wheel 341544 Apr 9 2014 /usr/lib/libssh.so.5
From a reboot email, the i386 host was on: Jan 12 16:24:11 gelt kernel: FreeBSD 9.3-RELEASE-p5 #0: Mon Nov 3 22:02:57 UTC 2014 before these commands were run: freebsd-update fetch install shutdown -r now
The problem seems to stem from a previous minor upgrade (9.2 -> 9.3) being incomplete. The kernel was upgraded but the userland was not, so /usr/lib/private/ didn't exist (as there wasn't one in 9.2)
This properly upgraded my host: env UNAME_r=9.2-RELEASE freebsd-update upgrade -r 9.3-RELEASE freebsd-update install freebsd-update install and perhaps a third: freebsd-update install
That workaround indeed fixed the broken host, and none of the other machines had a 9.2 /bin/sh, and furthermore none had any issues with the upgrade to 9.3R-p8. Thanks to all for finding the cause! I'd say this ticket can be closed.
*** Bug 195568 has been marked as a duplicate of this bug. ***
A fix to prevent this issue has been committed to -CURRENT https://reviews.freebsd.org/D1550
Do we need an EN for this?
(In reply to Xin LI from comment #16) This is a result of user error, so I am not sure it needs an EN. The fix is just a safety belt to keep you from shooting off your own foot.
(In reply to Xin LI from comment #16) > Do we need an EN for this? IMHO, yes, and EN should be issued for this. I'll handle the "paperwork" for it.
(In reply to Allan Jude from comment #15) > A fix to prevent this issue has been committed to -CURRENT > Which svn revision resolves this?
A commit references this bug: Author: allanjude Date: Tue Mar 3 23:20:19 UTC 2015 New revision: 279571 URL: https://svnweb.freebsd.org/changeset/base/279571 Log: Add a new safetly belt to freebsd-update to prevent a user doing a minor update (-pX) while having an unfinished major upgrade (9.x to 9.y) Safetly belt can be disabled with the -F flag Additionally, add the --not-running-from-cron flag they bypasses the TTY requirement, and allows freebsd-update to be invoked by orchestration frameworks, scripts, or otherwise. PR: 196760 Differential Revision: https://reviews.freebsd.org/D1550 Reviewed by: cperciva, delphij Approved by: bcr (mentor), rodrigc (src) MFC after: 1 month Relnotes: yes Sponsored by: ScaleEngine Inc. Changes: head/usr.sbin/freebsd-update/freebsd-update.8 head/usr.sbin/freebsd-update/freebsd-update.sh
(In reply to Glen Barber from comment #19) r279571
Reassign, pending EN.
A commit references this bug: Author: delphij Date: Wed May 13 22:36:01 UTC 2015 New revision: 282870 URL: https://svnweb.freebsd.org/changeset/base/282870 Log: MFC r279571,281563 (allanjude): Add a new safetly belt to freebsd-update to prevent a user doing a minor update (-pX) while having an unfinished major upgrade (9.x to 9.y) Safetly belt can be disabled with the -F flag Additionally, add the --not-running-from-cron flag they bypasses the TTY requirement, and allows freebsd-update to be invoked by orchestration frameworks, scripts, or otherwise. PR: 196760 Relnotes: yes Sponsored by: ScaleEngine Inc. Changes: _U stable/10/ stable/10/usr.sbin/freebsd-update/freebsd-update.8 stable/10/usr.sbin/freebsd-update/freebsd-update.sh
A commit references this bug: Author: delphij Date: Wed May 13 22:36:53 UTC 2015 New revision: 282871 URL: https://svnweb.freebsd.org/changeset/base/282871 Log: MFC r279571,281563 (allanjude): Add a new safetly belt to freebsd-update to prevent a user doing a minor update (-pX) while having an unfinished major upgrade (9.x to 9.y) Safetly belt can be disabled with the -F flag Additionally, add the --not-running-from-cron flag they bypasses the TTY requirement, and allows freebsd-update to be invoked by orchestration frameworks, scripts, or otherwise. PR: 196760 Changes: _U stable/9/usr.sbin/freebsd-update/ stable/9/usr.sbin/freebsd-update/freebsd-update.8 stable/9/usr.sbin/freebsd-update/freebsd-update.sh
This is fixed in FreeBSD-EN-15:04.freebsd-update.
*** Bug 199669 has been marked as a duplicate of this bug. ***