Bug 197578 - md5 returns random values each time; breaking freebsd-update
Summary: md5 returns random values each time; breaking freebsd-update
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.1-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-12 18:39 UTC by klamothe
Modified: 2017-08-10 14:49 UTC (History)
1 user (show)

See Also:


Attachments
debug output (66.91 KB, text/plain)
2015-02-12 19:30 UTC, klamothe
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description klamothe 2015-02-12 18:39:45 UTC
Hello,

I'm having an odd problem at times with this one server.

It started off with this. It's currently running 10.1-p5 on amd64 hardware. memtest86 yields no errors.

# freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 10.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files...
File changed while FreeBSD Update running: /boot/kernel/kernel

Running md5 on this file shows it's constantly changing

root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = 0aa7bfe32ca8bb34cbf31a3c988da062
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = 7736696cb6e9d90f80fbaf1b267d48bb
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = ccfd4eef7dd5f0862de3f502af244112
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # md5 /boot/kernel/kernel
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354

The odd thing is, using openssl with md5 for a checksum yields the same result everytime.

root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
root@monitor:~ # openssl dgst -md5 /boot/kernel/kernel
MD5(/boot/kernel/kernel)= eb30ceb29a80b57ba9609e843af56354
Comment 1 klamothe 2015-02-12 18:44:01 UTC
Extended md5 debugging:

# md5 -xt /boot/kernel/kernel
MD5 test suite:
MD5 ("") = d41d8cd98f00b204e9800998ecf8427e - verified correct
MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 - verified correct
MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72 - verified correct
MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0 - verified correct
MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b - verified correct
MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = d174ab98d277d9f5a5611c2c9f419d9f - verified correct
MD5 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 57edf4a22be3c955ac49da2e2107b67a - verified correct
MD5 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = b50663f41d44d92171cb9976bc118538 - verified correct
MD5 time trial. Digesting 100000 10000-byte blocks ... done
Digest = 766a2bb5d24bddae466c572bcabca3ee
Time = 2.046569 seconds
Speed = 488622624.000000 bytes/second
MD5 (/boot/kernel/kernel) = eb30ceb29a80b57ba9609e843af56354




# md5 -xt /boot/kernel/kernel
MD5 test suite:
MD5 ("") = d41d8cd98f00b204e9800998ecf8427e - verified correct
MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 - verified correct
MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72 - verified correct
MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0 - verified correct
MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b - verified correct
MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = d174ab98d277d9f5a5611c2c9f419d9f - verified correct
MD5 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 57edf4a22be3c955ac49da2e2107b67a - verified correct
MD5 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = b50663f41d44d92171cb9976bc118538 - verified correct
MD5 time trial. Digesting 100000 10000-byte blocks ... done
Digest = 766a2bb5d24bddae466c572bcabca3ee
Time = 2.021598 seconds
Speed = 494658176.000000 bytes/second
MD5 (/boot/kernel/kernel) = de43a68b17ce556838b2e02656783909
Comment 2 Glen Barber freebsd_committer 2015-02-12 19:00:17 UTC
Can you please provide the following output?

 pciconf -lv
 sysctl kern
 sysctl vfs
 sysctl dev
 sysctl hw
Comment 3 klamothe 2015-02-12 19:30:34 UTC
Created attachment 152917 [details]
debug output
Comment 4 Glen Barber freebsd_committer 2015-02-12 20:14:35 UTC
Will you try adding 'vfs.unmapped_buf_allowed=0' to /boot/loader.conf, and rebooting the system?  There were reports of having vfs.unmapped_buf_allowed enabled resulting in similar behavior in hypervisors (VirtualBox, Xen), but to my knowledge this was limited to i386.
Comment 5 klamothe 2015-02-12 20:34:30 UTC
Added that to loader.conf and restarted the server.

Still the same problem.

Oddly enough the other bundled tools that come with the base system such as sha1, sha256, sha512 and rmd160 do not have this behaviour either...
Comment 6 Glen Barber freebsd_committer 2015-07-08 13:17:24 UTC
Can you verify if you still see this problem on 10.2-PRERELEASE (although, freebsd-update(8) will not be able to update to that)?
Comment 7 Ed Maste freebsd_committer 2017-08-10 14:49:49 UTC
Note that the message you observed is based on comparing output of sha256, not md5:

                # Make sure the file hasn't changed.
                cp "${BASEDIR}/${F}" tmpfile
                if [ `sha256 -q tmpfile` != ${HASH} ]; then
                        echo
                        echo "File changed while FreeBSD Update running: ${F}"
                        return 1
                fi