Bug 195990
| Summary: | freebsd-update not upgrading files properly | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Jeroen <freebsd> | ||||
| Component: | bin | Assignee: | freebsd-bugs (Nobody) <bugs> | ||||
| Status: | Closed Overcome By Events | ||||||
| Severity: | Affects Some People | CC: | Jungleboogie0, cperciva, emaste, grahamperrin, joshruehlig, linimon, naruse, rtd24 | ||||
| Priority: | --- | ||||||
| Version: | 10.1-RELEASE | ||||||
| Hardware: | Any | ||||||
| OS: | Any | ||||||
| Attachments: |
|
||||||
|
Description
Jeroen
2014-12-15 10:12:58 UTC
This seems caused by freebsd-update's FreeBSD-SA-14:28.file patch. It updates file(1)'s version from 11 to 19, and it conflicts magic data. No follow-ups in a week. Is there an another correct place to report? This is the correct place to report. Many people are inactive for the holidays. Browsing through github it looks like some regression was fixed regarding this back in October 2014 (r273688). This fix is only on master and was never backported to stable/10 even though the regression exists there. This bug is also affecting my php application which uses 'file'. Though joshruehlig investigated r273688 fixes this, it's not bugported yet and of course FreeBSD 10.1-p4 doesn't include the fix. What's the best way to get this regression fixed in stable/10? @delphij continues to push file forward in stable/10 but missed applying r273688 as he did to master (stable/11) https://github.com/freebsd/freebsd/commit/ed6a1857098c893243cd94f168fadd1832ef6fc4 I could try applying the changed myself and submitting a pull request if that would help. Thanks, Minor update: this is still broken for file-5.22 and 10.1-RELEASE-p5. This also breaks py-magic, which is what led me here. Try: >>> import magic >>> m = magic.open(magic.MAGIC_MIME) >>> m.load() Result: ...same errors as with file(1), ending with /usr/share/misc/magic, 17382: Warning: Printf format `l' is not valid for type `byte' in description `- version %ld' Thanks for the workaround. For fixing py27-magic, I can pass it the port version's magic file instead, like so: >>> m.load('/usr/local/share/file/magic') 0 >>> m.file('.macromedia') 'inode/directory; charset=binary' My work around is to use pkg file since that works. I guess base won't be updated?? Maybe 10.2 will have an updated file. % uname -a FreeBSD lyander-fbsd 10.1-RELEASE-p5 FreeBSD 10.1-RELEASE-p5 #0: Tue Jan 27 08:52:50 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 % freebsd-version -k 10.1-RELEASE-p5 % freebsd-version -u 10.1-RELEASE-p5 % file --version file-5.11 magic file from /usr/share/misc/magic Installing file pkg: % /usr/local/bin/file --version file-5.22 magic file from /usr/local/share/file/magic % /usr/local/bin/file dns.sh dns.sh: ASCII text This is not fixed yet. delphij@, please see comment 6. It seems you may be the correct person to decide the way forwards. (In reply to Jeroen from comment #0) I've verified that the file shipped with 10.1-RELEASE was of version 12. I believe there was something wrong with the update procedure, can you run freebsd-update IDS and see if there is any corruption? (In reply to Xin LI from comment #11) Here is a FreeBSD/amd64 10.1-RELEASE system which was updated from 10.0-RELEASE with freebsd-update. root@www:/root # which file /usr/bin/file root@www:/root # file --version file-5.19 magic file from /usr/share/misc/magic root@www:/root # file Usage: file [-bcEhikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type] [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ... file -C [-m magicfiles] file [--help] root@www:/root # uname - root@www:/root # freebsd-version -k 10.1-RELEASE-p6 root@www:/root # freebsd-version -u 10.1-RELEASE-p8 (In reply to joshruehlig from comment #4) The change is part of the update to 5.21 change. It was not applied to releng/10.1 because it was believed that the problem does not affect a majority of users. I'm not in objection of making an errata for 10.1-RELEASE and update file to the latest version though. (In reply to Ryder from comment #7) Can't reproduce: root@www:/root # pkg install py27-magic Updating FreeBSD repository catalogue... FreeBSD repository is up-to-date. All repositories are up-to-date. The following 1 packages will be affected (of 0 checked): New packages to be INSTALLED: py27-magic: 5.18 The process will require 23 KiB more space. 4 KiB to be downloaded. Proceed with this action? [y/N]: y Fetching py27-magic-5.18.txz: 100% 4 KiB 4.6kB/s 00:01 Checking integrity... done (0 conflicting) [1/1] Installing py27-magic-5.18... [1/1] Extracting py27-magic-5.18: 100% root@www:/root # python Python 2.7.9 (default, Feb 25 2015, 07:23:55) [GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10 Type "help", "copyright", "credits" or "license" for more information. >>> import magic >>> m = magic.open(magic.MAGIC_MIME) >>> m.load() 0 >>> (In reply to Xin LI from comment #13) Whatever you think is best. It would be great to be able to get the warning messages to go away on 10.1 =] (In reply to Xin LI from comment #14) I can only reproduce this error on machines which used an update path. I've just set up a new 10.1-RELEASE-p6 (from an image provided by cperciva for FreeBSD on EC2), and there the default installation of file-5.19 doesn't produce the Jeroen's error that started this thread, nor does it affect the python module. It seems as if the correct /usr/share/misc/magic file is part of the installation, but this file is not patched when taking the update path. (In reply to Xin LI from comment #14) As I wrote in #1, this is caused by freebsd-update's FreeBSD-SA-14:28.file patch. (In reply to naruse from comment #17) Hi, I have applied the change with freebsd-update and was not able to reproduce it, the file is in its due form. Could you please run freebsd-update IDS and see if the files are all original? When I ran 'freebsd update IDS' in my ezjail which is actually affected by this bug, all my files showed as being modified, so I don't think this works for ezjails. sha256sum for magic that was in my jail was 00fa55a6721024d8ca0da78cf92184b968a55f52ef50584adc7e0572c2b63c5d sha256sum for magic available from here http://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/10.1-STABLE/ was 1229764bf991c40a8b5beb8b23e40638749aa4ddcba267f00ac6f91328a5a2e0 Replacing /usr/jails/basejail/usr/share/misc/magic seemed to have fixed this issue for me. (In reply to joshruehlig from comment #19) Well, you can't verify your -STABLE system with freebsd-update. My impression is that this is probably because either an ezjail bug or a partial update: all systems I have, regardless I use the traditional 'make buildworld buildkernel installkernel installworld' or freebsd-update, or 10.1-RELEASE or 10.1-STABLE, I have never seen this happen and I was unable to reproduce the problem you have seen. (In reply to Xin LI from comment #20) Whoops, I linked the wrong spot. I'm not on STABLE, my host + ezjail is on 10.1-p8. 'freebsd-update IDS' seemed to work on my host (only showed a few files that had been midified, most of which were etc's I edited) But, it fails miserably in ezjail. Created attachment 155294 [details]
IDS output of FreeBSD 10.1 amd64
Attached is my FreeBSD 10.1 amd64's IDS output, which is initially 8.2 and upgraded with freebsd-update.
(In reply to naruse from comment #22) Hi, This suggests that you have incomplete/partial installation of 10.1-RELEASE update. For instance 55ffd3b705b6ad848f900eb642cdbceb5c7b039b84a43be2fd3a2da37fd489ae is 10.0-RELEASE's chflag(1). Did you hit some errors when installing the update or did you force it to stop? I really can't reproduce it locally but if it's a bug with freebsd-update we definitely need to fix it. (In reply to Xin LI from comment #23) > Did you hit some errors when installing the update or did you force it to stop? As far as I remember, no. Moreover I have 3 FreeBSD system and all of them affect this. (All of them are upgraded from FreeBSD 8.2 or earlier, i386/amd64) The output of `freebsd-update IDS` for my system is a log file of 8.6MB.... I have had some minor problems with freebsd-update before such as bug #196055, but don't recall any other significant problem. (In reply to Jeroen from comment #25) Well, I think this indicates that freebsd-update is not upgrading files properly, so I'm changing the topic of this bug to reflect that. This has nothing to do with file -- you are seeing the issue because the data files were not updated as part of a freebsd-update upgrade. Colin, do you have some suggestions on what we should look at next? If one can reliably trigger the problem we would need to find out why and inform the user community how to avoid these issue. The only options I can guess are 1. The world updates never got installed after the kernel updates and rebooting. (A patch recently went into the tree to warn about such partial updates.) 2. The recently-fixed UFS bug which was causing hanging on reboots also resulted in filesystem operations being left unsynchronized, causing files to be "uninstalled". Either way, I don't know that there's anything we can do at this point... (In reply to Colin Percival from comment #27) Is there a way that I can help you confirm that that is what happened to my system? Drop freebsd-update PRs which were assigned to me. I'm not working on this code any more. I am sorry that we were unable to confirm the issues that occurred during this update. The referenced versions are now well past EOL so closing this ticket. |
