Last week the possible removal of MD5 checksums in favor of SHA256 checksums from distinfo was discussed on #bsdports. dougb@ already spotted some ports without a SHA256 checksum and reported them. The consensus was that SHA256 is a stronger algorithm than MD5, making the latter redundant. so@ and/or ports-security@ might also be interested in this change request. This patch to bsd.port.mk signals if a distinfo file contains MD5 lines and gives a non-fatal notice about it. It removes md5 from CHECKSUM_ALGORITHMS, so no new MD5 checksums are being generated, nor are they checked. While here, rename MD5_FILE to DISTINFO_FILE. Fix: Patch attached with submission follows:
Responsible Changed From-To: freebsd-ports-bugs->portmgr bsd.port.mk is portmgr territory (via the GNATS Auto Assign Tool)
I'm not positive that emitting a warning for an MD5 in a file is necessary, my personal preference is that we just silently ignore them and remove the lines at our leisure. However the rest of the patch looks really good, thanks for doing this! The status on the remaining ports with no SHA256: Marked DEPRECATED with expiration 9/1/2010 on 8/8/2010: ports/editors/staroffice52 ports/editors/staroffice60 ports/editors/staroffice70 ports/french/staroffice52 ports/german/staroffice52 ports/german/staroffice70 ports/portuguese/staroffice52 Marked DEPRECATED with expiration 9/1/2010 on 8/8/2010 (separate commits): ports/java/simplicity ports/lang/icc7 ports/149449: ports/databases/sybase-ocsd ports/149450: ports/editors/staroffice5 ports/149452: ports/emulators/linux-vmware-toolbox2 ports/149453: ports/german/staroffice5 ports/149455: ports/security/tripwire-131 ports/149456: ports/sysutils/linux-acu From cy: "I will commit a fix after I return, after Aug 13." For the ports with PRs I plan to DEPRECATE them on 8/22 with the same expiration date. They aren't fetchable, so it shouldn't matter. Alternately, I could mark them DEPRECATED now and let the maintainers un-mark them if they choose to fix them. hth, Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso
The idea of the warning is to make remaining MD5 lines more visible. I just noticed that the grep expression on line 4895 is wrong, it should read: if ${GREP} ^MD5 ${DISTINFO_FILE} > /dev/null; then \ (note the extra '^' in front of 'MD5') This is to prevent giving false positives on distfiles which have 'MD5' in their name. Rene
While SHA256 is considered stronger I don't think redundancy is bad. And it makes easier to check MD5 against upstream .md5sum file for corruption. BTW, are there any logs of the discussion on #bsdports? I think the history of descision making is important for an *open* community.
2010/8/15 Anonymous <swell.k@gmail.com>: > While SHA256 is considered stronger I don't think redundancy is bad. > And it makes easier to check MD5 against upstream .md5sum file for > corruption. > MD5 should be avoided for hashing since it is susceptible to collision attacks. See also cperciva@'s last talk at BSDCan: http://www.bsdcan.org/2010/schedule/attachments/135_crypto1hr.pdf > BTW, are there any logs of the discussion on #bsdports? I think the > history of descision making is important for an *open* community. > My client (irssi) doesn't store them, sorry. For now nothing is decided yet about removing or even deprecating them. I sent the PR exactly for publishing the idea on #bsdports (on efnet). If I remember correctly the history was like "hey, some ports don't have SHA256 checksums" and "well, but if all ports have them, why do we need MD5 checksums, since they are 'inferior'. So maybe we can remove them some day".
Ren=C3=A9 Ladan <rene@freebsd.org> writes: > 2010/8/15 Anonymous <swell.k@gmail.com>: >> While SHA256 is considered stronger I don't think redundancy is bad. >> And it makes easier to check MD5 against upstream .md5sum file for >> corruption. >> > MD5 should be avoided for hashing since it is susceptible to collision > attacks. See also cperciva@'s last talk at BSDCan: > http://www.bsdcan.org/2010/schedule/attachments/135_crypto1hr.pdf That presentation doesn't mention redundancy in the security onion. One would have to find a collision that affects both MD5 and SHA256 at the same time in order to forge a fake distfile. Why not just fail if *any* of the used algos is not present in distinfo? --- a.diff begins here --- Index: Mk/bsd.port.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /a/.cvsup/ports/Mk/bsd.port.mk,v retrieving revision 1.643 diff -u -p -r1.643 bsd.port.mk --- Mk/bsd.port.mk 15 Jul 2010 14:48:50 -0000 1.643 +++ Mk/bsd.port.mk 15 Aug 2010 13:14:04 -0000 @@ -4905,7 +4912,7 @@ checksum: fetch check-checksum-algorithm \ if [ $$ignore =3D "false" -a -z "$$CKSUM" ]; then \ ${ECHO_MSG} "=3D> No $$alg checksum recorded for $$file."; \ - ignore=3D"true"; \ + exit 1; \ fi; \ \ if [ "$$CKSUM" =3D "IGNORE" ]; then \ --- a.diff ends here ---
On Sun, 15 Aug 2010, René Ladan wrote: > The idea of the warning is to make remaining MD5 lines more visible. Yes, I understand what "warning" means. :) What I'm saying is, I don't care if files still have MD5 lines. As long as they are not used, they cannot hurt anything. Also, in that same discussion on IRC there were several discussions about methodology for removing them, so my first choice would be to have bsd.port.mk silently ignore them while we get volunteers to remove them one category at a time. But either way, that's the smallest bit of it, simply moving forward on using SHA256 only is a great step. :) Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso
I had 2 PR numbers reversed in my previous post, sorry. The status of the last 2 is: ports/149455: ports/sysutils/linux-acu - No response from MAINTAINER ports/149456: ports/security/tripwire-131 From cy: "I will commit a fix after I return, after Aug 13." *** DONE *** As for the history, the plan all along was to replace md5 with sha256. The reason it wasn't done all at once was that we still had supported versions of FreeBSD which did not contain a sha256 binary at the time this feature was introduced to bsd.port.mk. I will leave it as an exercise for the reader to educate themselves on why having both doesn't provide any extra security, and why transitioning to sha256 is the right way to go. Also, if you're concerned about checking the .md5sum file somewhere then there is no reason you can't do that yourself, the ports infrastructure doesn't need to do it for you. :) The discussion on IRC went like this: <me> Isn't it about time we removed MD5 from distinfo? <others> Do all supported versions of FreeBSD have sha256 now? <me> yes <others> Do all ports have SHA256 lines? <me> good question! <me> turns out that almost all of them do, and I've taken care of the ones that don't <others> let's go for it! hth, Doug
Doug Barton <dougb@dougbarton.us> writes: > I will leave it as an exercise for the reader to educate themselves on > why having both doesn't provide any extra security, and why > transitioning to sha256 is the right way to go. If it doesn't affect security negatively[1] then why bother? I don't see the point of extra churn that affects practically every distinfo for no real benefit. [1] unless you're trying to shift blame from bsd.port.mk that it ignores missing checksum lines in distinfo on MD5 being insecure
On 08/20/2010 17:03, Anonymous wrote: > Doug Barton<dougb@dougbarton.us> writes: > >> I will leave it as an exercise for the reader to educate themselves on >> why having both doesn't provide any extra security, and why >> transitioning to sha256 is the right way to go. > > If it doesn't affect security negatively[1] then why bother? I don't see > the point of extra churn that affects practically every distinfo for no > real benefit. 1. It was always intended to be removed when it was no longer necessary. 2. Less code complexity is a good thing, especially for no added benefit. Doug -- Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso
Doug Barton <dougb@dougbarton.us> writes: > On 08/20/2010 17:03, Anonymous wrote: >> Doug Barton<dougb@dougbarton.us> writes: >> >>> I will leave it as an exercise for the reader to educate themselves on >>> why having both doesn't provide any extra security, and why >>> transitioning to sha256 is the right way to go. >> >> If it doesn't affect security negatively[1] then why bother? I don't see >> the point of extra churn that affects practically every distinfo for no >> real benefit. > > 1. It was always intended to be removed when it was no longer necessary. Well, it could be done along with adding new algorithm to CHECKSUM_ALGORITHMS or replacing sha256 with smth better. > 2. Less code complexity is a good thing, especially for no added benefit. Where? Perhaps I'm looking at the wrong diff but all it does - rename MD5_FILE -> DISTINFO_FILE - replace check in do-fetch to rely on SHA256 line - remove `md5' from CHECKSUM_ALGORITHMS - add noisy warning about MD5 being deprecated instead of complaining that distinfo contains unused cruft and displaying it Missing checksums are still shoved under carpet except when there is only one algorithm. And the code still expects more than one algorithm in CHECKSUM_ALGORITHMS.
FYI, all ports without SHA256 checksums in their distinfo file have now been removed. Doug
Given that nobody likes the verbose warning when an MD5 line is present, I suggest to remove it. @swell.k: I fail to see the difference your a.diff patch makes? If I omit the SHA256 line from a distinfo, bsd.port.mk bails out complaining that there is no SHA256 line both with and without your patch. Also, having support for multiple checksum algorithms is good because when SHA256 is superseded by some new algorithm (SHA3), the new checksums can just be added without breaking the entire tree. Somehow bsd.port.mk fails to complain when I force a size mismatch, which is caused by the checksum target not checking for it, only the fetch target does this. Maybe this should be added? Having the size check also in the checksum target lessens the possibility of a checksum collision.
On 09/05/2010 08:12 AM, René Ladan wrote: > Given that nobody likes the verbose warning when an MD5 line is > present, I suggest to remove it. Agreed. > Somehow bsd.port.mk fails to complain when I force a size mismatch, > which is caused by the checksum target not checking for it, only the > fetch target does this. Maybe this should be added? Having the size > check also in the checksum target lessens the possibility of a > checksum collision. The chances of a meaningful collision with SHA256 are so incredibly small that I don't think it's worth worrying about. OTOH, if you want to add code to bsd.port.mk to check that as part of the checksum target I wouldn't object. Doug
Patch looks ok to me on principle (after the warning has been removed). I haven't checked but make sure no port uses MD5_FILE directly as it's being renamed to DISTINFO_FILE. I'm away so someone else will have to run it through pointyhat if it reaches consensus. -- Florent Thoumie flz@FreeBSD.org FreeBSD Committer
On 27-09-2010 08:52, Florent Thoumie wrote: > Patch looks ok to me on principle (after the warning has been removed). > > I haven't checked but make sure no port uses MD5_FILE directly as it's > being renamed to DISTINFO_FILE. > > I'm away so someone else will have to run it through pointyhat if it > reaches consensus. > I have attached the list of files which have MD5_FILE in them, these include ports-mgmt/portmk/Mk/bsd.port.mk which the patch does not address.
Attached is the patch for converting all instances of MD5_FILE to DISTINFO_FILE for the ports tree of today, except those in Mk/bsd.port.mk (addressed in the patches above) and those in ports-mgmt/portmk/Mk/bsd.port.mk
State Changed From-To: open->analyzed Take patch.txt and patch-3 for an -exp
patch for emulators category, which was missing in patch-3.diff
State Changed From-To: analyzed->closed Committed, thanks!
erwin 2010-10-28 21:00:21 UTC FreeBSD ports repository Modified files: Mk bsd.linux-rpm.mk bsd.port.mk bsd.qt.mk accessibility/ktts Makefile astro/marble Makefile audio/festvox-us1-mbrola Makefile audio/py-musicbrainz Makefile chinese/acroread8-zh_CN Makefile chinese/acroread8-zh_TW Makefile chinese/joe Makefile chinese/joe2 Makefile databases/db41-nocrypto Makefile databases/db42-nocrypto Makefile databases/linux-oracle-instantclient-basic Makefile databases/linux-oracle-instantclient-sdk Makefile databases/linux-oracle-instantclient-sqlplus Makefile databases/mysql-workbench52 Makefile databases/py-bsddb Makefile databases/py-gdbm Makefile databases/py-sqlite3 Makefile databases/qt4-sqlite3-plugin Makefile deskutils/multisync-backup Makefile deskutils/multisync-syncml Makefile devel/glib20-reference bsd.gnome-reference.mk devel/linux-allegro Makefile devel/p4 Makefile devel/p4d Makefile devel/p4ftpd Makefile devel/p4p Makefile devel/p4v Makefile devel/p4web Makefile devel/subversion-freebsd Makefile emulators/linux_base-f10 Makefile emulators/linux_base-f7 Makefile emulators/linux_base-f8 Makefile emulators/linux_base-f9 Makefile emulators/linux_base-fc4 Makefile emulators/linux_base-fc6 Makefile emulators/mupen64-base Makefile.common french/acroread8 Makefile french/acroread9 Makefile french/aster bsd.aster.mk french/mozilla-flp Makefile games/gnushogi Makefile.common german/acroread8 Makefile german/acroread9 Makefile graphics/linux-gdk-pixbuf Makefile graphics/linux-sdl_image Makefile japanese/acroread8 Makefile japanese/acroread9 Makefile japanese/ical Makefile japanese/kanji26 Makefile japanese/p5-Text-ChaSen Makefile japanese/roundcube Makefile japanese/ruby-man Makefile korean/acroread8 Makefile lang/php_doc Makefile math/dislin Makefile math/vtk-data Makefile misc/dnetc Makefile net/AquaGatekeeper Makefile net/AquaGatekeeper2 Makefile net/iwi-firmware Makefile net/iwi-firmware-kmod Makefile net-im/licq Makefile.inc net-p2p/libtorrent-devel Makefile net-p2p/rtorrent-devel Makefile ports-mgmt/portlint/src portlint.pl portuguese/acroread8 Makefile print/bjfiltercom Makefile print/dk-acroread8 Makefile print/es-acroread8 Makefile print/fi-acroread8 Makefile print/it-acroread8 Makefile print/kdeutils4-printer-applet Makefile print/nl-acroread8 Makefile print/no-acroread8 Makefile print/pips800 Makefile.pips print/sv-acroread8 Makefile print/system-config-printer-kde Makefile security/cyrus-sasl2-ldapdb Makefile security/cyrus-sasl2-saslauthd Makefile sysutils/rsyslog3 bsd.rsyslog.mk sysutils/rsyslog4 Makefile sysutils/rsyslog4-devel Makefile sysutils/rsyslog5 Makefile sysutils/rsyslog5-devel Makefile sysutils/rsyslog6-devel Makefile textproc/jdictionary Makefile.plugin textproc/linux-aspell Makefile textproc/linux-scim-gtk Makefile textproc/linux-scim-libs Makefile textproc/py-expat Makefile www/mod_pubcookie Makefile www/openacs-dotlrn Makefile x11/plasma-scriptengine-python Makefile x11/plasma-scriptengine-ruby Makefile x11/ruby-gnome2 Makefile.common x11-themes/gtk-aluminumalloy-cryogenic-theme Makefile x11-themes/gtk-aluminumalloy-smog-theme Makefile x11-themes/gtk-aluminumalloy-toxic-theme Makefile x11-toolkits/linux-f10-openmotif Makefile x11-toolkits/linux-f8-openmotif Makefile x11-toolkits/py-tkinter Makefile x11-wm/libwraster Makefile Log: Deprecate md5 in favour of sha256 checksums. md5 checksums will no longer be generated or checked, and will be silently ignored for now. Also, generalize the MD5_FILE macro to DISTINFO_FILO. PR: 149657 Submitted by: rene Approved by: portmgr Tested on: pointyhat i386 7-exp Revision Changes Path 1.24 +2 -2 ports/Mk/bsd.linux-rpm.mk 1.652 +24 -25 ports/Mk/bsd.port.mk 1.19 +2 -2 ports/Mk/bsd.qt.mk 1.4 +1 -1 ports/accessibility/ktts/Makefile 1.5 +1 -1 ports/astro/marble/Makefile 1.6 +1 -1 ports/audio/festvox-us1-mbrola/Makefile 1.17 +1 -1 ports/audio/py-musicbrainz/Makefile 1.2 +1 -1 ports/chinese/acroread8-zh_CN/Makefile 1.2 +1 -1 ports/chinese/acroread8-zh_TW/Makefile 1.27 +1 -1 ports/chinese/joe/Makefile 1.24 +1 -1 ports/chinese/joe2/Makefile 1.3 +1 -1 ports/databases/db41-nocrypto/Makefile 1.4 +1 -1 ports/databases/db42-nocrypto/Makefile 1.10 +1 -1 ports/databases/linux-oracle-instantclient-basic/Makefile 1.7 +1 -1 ports/databases/linux-oracle-instantclient-sdk/Makefile 1.8 +1 -1 ports/databases/linux-oracle-instantclient-sqlplus/Makefile 1.3 +1 -1 ports/databases/mysql-workbench52/Makefile 1.15 +1 -1 ports/databases/py-bsddb/Makefile 1.17 +1 -1 ports/databases/py-gdbm/Makefile 1.8 +1 -1 ports/databases/py-sqlite3/Makefile 1.14 +1 -1 ports/databases/qt4-sqlite3-plugin/Makefile 1.4 +1 -1 ports/deskutils/multisync-backup/Makefile 1.4 +1 -1 ports/deskutils/multisync-syncml/Makefile 1.9 +2 -2 ports/devel/glib20-reference/bsd.gnome-reference.mk 1.14 +1 -1 ports/devel/linux-allegro/Makefile 1.88 +1 -1 ports/devel/p4/Makefile 1.90 +1 -1 ports/devel/p4d/Makefile 1.87 +1 -1 ports/devel/p4ftpd/Makefile 1.87 +1 -1 ports/devel/p4p/Makefile 1.30 +1 -1 ports/devel/p4v/Makefile 1.87 +1 -1 ports/devel/p4web/Makefile 1.140 +1 -1 ports/devel/subversion-freebsd/Makefile 1.58 +1 -1 ports/emulators/linux_base-f10/Makefile 1.42 +1 -1 ports/emulators/linux_base-f7/Makefile 1.54 +1 -1 ports/emulators/linux_base-f8/Makefile 1.57 +1 -1 ports/emulators/linux_base-f9/Makefile 1.34 +1 -1 ports/emulators/linux_base-fc4/Makefile 1.37 +1 -1 ports/emulators/linux_base-fc6/Makefile 1.7 +2 -2 ports/emulators/mupen64-base/Makefile.common 1.2 +1 -1 ports/french/acroread8/Makefile 1.2 +1 -1 ports/french/acroread9/Makefile 1.2 +2 -2 ports/french/aster/bsd.aster.mk 1.40 +1 -1 ports/french/mozilla-flp/Makefile 1.3 +2 -2 ports/games/gnushogi/Makefile.common 1.2 +1 -1 ports/german/acroread8/Makefile 1.2 +1 -1 ports/german/acroread9/Makefile 1.15 +1 -1 ports/graphics/linux-gdk-pixbuf/Makefile 1.17 +1 -1 ports/graphics/linux-sdl_image/Makefile 1.2 +1 -1 ports/japanese/acroread8/Makefile 1.2 +1 -1 ports/japanese/acroread9/Makefile 1.10 +1 -1 ports/japanese/ical/Makefile 1.6 +2 -2 ports/japanese/kanji26/Makefile 1.14 +1 -1 ports/japanese/p5-Text-ChaSen/Makefile 1.3 +1 -1 ports/japanese/roundcube/Makefile 1.7 +1 -1 ports/japanese/ruby-man/Makefile 1.2 +1 -1 ports/korean/acroread8/Makefile 1.41 +3 -4 ports/lang/php_doc/Makefile 1.45 +1 -1 ports/math/dislin/Makefile 1.6 +1 -1 ports/math/vtk-data/Makefile 1.67 +1 -1 ports/misc/dnetc/Makefile 1.28 +2 -2 ports/net-im/licq/Makefile.inc 1.27 +1 -1 ports/net-p2p/libtorrent-devel/Makefile 1.31 +1 -1 ports/net-p2p/rtorrent-devel/Makefile 1.16 +1 -1 ports/net/AquaGatekeeper/Makefile 1.16 +1 -1 ports/net/AquaGatekeeper2/Makefile 1.7 +1 -1 ports/net/iwi-firmware-kmod/Makefile 1.11 +1 -1 ports/net/iwi-firmware/Makefile 1.115 +5 -5 ports/ports-mgmt/portlint/src/portlint.pl 1.2 +1 -1 ports/portuguese/acroread8/Makefile 1.29 +1 -1 ports/print/bjfiltercom/Makefile 1.2 +1 -1 ports/print/dk-acroread8/Makefile 1.2 +1 -1 ports/print/es-acroread8/Makefile 1.2 +1 -1 ports/print/fi-acroread8/Makefile 1.2 +1 -1 ports/print/it-acroread8/Makefile 1.13 +1 -1 ports/print/kdeutils4-printer-applet/Makefile 1.2 +1 -1 ports/print/nl-acroread8/Makefile 1.2 +1 -1 ports/print/no-acroread8/Makefile 1.39 +2 -2 ports/print/pips800/Makefile.pips 1.2 +1 -1 ports/print/sv-acroread8/Makefile 1.13 +1 -1 ports/print/system-config-printer-kde/Makefile 1.12 +1 -1 ports/security/cyrus-sasl2-ldapdb/Makefile 1.55 +1 -1 ports/security/cyrus-sasl2-saslauthd/Makefile 1.17 +1 -1 ports/sysutils/rsyslog3/bsd.rsyslog.mk 1.17 +1 -1 ports/sysutils/rsyslog4-devel/Makefile 1.16 +1 -1 ports/sysutils/rsyslog4/Makefile 1.22 +1 -1 ports/sysutils/rsyslog5-devel/Makefile 1.22 +1 -1 ports/sysutils/rsyslog5/Makefile 1.25 +1 -1 ports/sysutils/rsyslog6-devel/Makefile 1.3 +2 -2 ports/textproc/jdictionary/Makefile.plugin 1.17 +1 -1 ports/textproc/linux-aspell/Makefile 1.6 +1 -1 ports/textproc/linux-scim-gtk/Makefile 1.8 +1 -1 ports/textproc/linux-scim-libs/Makefile 1.19 +1 -1 ports/textproc/py-expat/Makefile 1.6 +1 -1 ports/www/mod_pubcookie/Makefile 1.17 +1 -1 ports/www/openacs-dotlrn/Makefile 1.7 +1 -1 ports/x11-themes/gtk-aluminumalloy-cryogenic-theme/Makefile 1.7 +1 -1 ports/x11-themes/gtk-aluminumalloy-smog-theme/Makefile 1.7 +1 -1 ports/x11-themes/gtk-aluminumalloy-toxic-theme/Makefile 1.21 +1 -1 ports/x11-toolkits/linux-f10-openmotif/Makefile 1.22 +1 -1 ports/x11-toolkits/linux-f8-openmotif/Makefile 1.21 +1 -1 ports/x11-toolkits/py-tkinter/Makefile 1.6 +1 -1 ports/x11-wm/libwraster/Makefile 1.2 +1 -1 ports/x11/plasma-scriptengine-python/Makefile 1.2 +1 -1 ports/x11/plasma-scriptengine-ruby/Makefile 1.36 +2 -2 ports/x11/ruby-gnome2/Makefile.common _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"