Many ports offer nowadays LZMA compressed distfiles (.tar.xz) that give (much) better compression than bzip2. The attached patch updates bsd.port.mk and bsd.commands.mk to support the USE_XZ flag for xz compressed files. http://tukaani.org/xz/
Responsible Changed From-To: freebsd-ports-bugs->portmgr bsd.port.mk is portmgr territory (via the GNATS Auto Assign Tool)
How many ports are we talking about here? I see you just added some lzma support to base system but I don't know whether it's compatible with archivers/xz. If it is, it might be worth changing the patch to use it. -- Florent Thoumie flz@FreeBSD.org FreeBSD Committer
Yes, the patch should actually detect if it is available in the base system and if not use the port. We are talking about many ports but not a killing number. I can give examples of ports distributing .xz or .lzma files: GNU is adopting this widely: grep, patch, autoconf, automake, gawk, libtool, wget, ..... Others: ImageMagick, ghostscript, libpng, .... The main benefits for compressed source code are: a) it decompresses up twice as fast than bz2 (but always faster) b) it compresses much better than bz2 (I yield results of 30% and better in my tests) (but always better)
On Tue, May 11, 2010 at 8:05 PM, Martin Matuska <mm@freebsd.org> wrote: > Yes, the patch should actually detect if it is available in the base > system and if not use the port. > > We are talking about many ports but not a killing number. > > I can give examples of ports distributing .xz or .lzma files: > GNU is adopting this widely: grep, patch, autoconf, automake, gawk, > libtool, wget, ..... > Others: ImageMagick, ghostscript, libpng, .... > > The main benefits for compressed source code are: > a) it decompresses up twice as fast than bz2 (but always faster) > b) it compresses much better than bz2 (I yield results of 30% and better > in my tests) (but always better) Beside updating the patch to use base system tools if available, could you follow-up with a patch to update a few ports to use USE_XZ? -- Florent Thoumie flz@FreeBSD.org FreeBSD Committer
I will start with ImageMagick right away and I would recommend ghostscript as well. DÅa 11. 5. 2010 21:14, Florent Thoumie wrote / napÃsal(a): > On Tue, May 11, 2010 at 8:05 PM, Martin Matuska <mm@freebsd.org> wrote: > >> Yes, the patch should actually detect if it is available in the base >> system and if not use the port. >> >> We are talking about many ports but not a killing number. >> >> I can give examples of ports distributing .xz or .lzma files: >> GNU is adopting this widely: grep, patch, autoconf, automake, gawk, >> libtool, wget, ..... >> Others: ImageMagick, ghostscript, libpng, .... >> >> The main benefits for compressed source code are: >> a) it decompresses up twice as fast than bz2 (but always faster) >> b) it compresses much better than bz2 (I yield results of 30% and better >> in my tests) (but always better) >> > Beside updating the patch to use base system tools if available, could > you follow-up with a patch to update a few ports to use USE_XZ? > >
http://people.freebsd.org/~mm/patches/xz/ports-lzma.patch
On Wed, May 12, 2010 at 9:39 PM, Martin Matuska <mm@freebsd.org> wrote: > http://people.freebsd.org/~mm/patches/xz/ports-lzma.patch Just mentioning that the patch contains the following changes that I discussed with Martin over IRC: - change USE_XZ to USE_LZMA because it'll be more meaningful. - change default EXTRACT_SUFX to .tar.lzma as it seems more common than .tar.xz. -- Florent Thoumie flz@FreeBSD.org FreeBSD Committer
Here are the patches, but som ports got updated since I wrote the patches: http://people.freebsd.org/~mm/patches/xz/ports-lzma.patch http://people.freebsd.org/~mm/patches/xz/ports-lzma-2.patch The main question is: USE_LZMA or USE_XZ? Some ports use .tar.xz, some .tar.lzma (both are extractable with xz)
Martin Matuska: > The main question is: > USE_LZMA or USE_XZ? > > Some ports use .tar.xz, some .tar.lzma (both are extractable with xz) Those are different container formats. (The container format provides a file(1) signature, integrity checks, etc.) Lasse Collin started the project under the name LZMA Utils and used a provisional container format; those are the .tar.lzma file you see. When he had finished designing the permanent file format (see http://tukaani.org/xz/format.html), he also changed the project name to XZ Utils and the file suffix to .xz. xz(1) can still extract the old .lzma files, but they should go away eventually. With this in mind, and speaking as the (ex-)maintainer of archivers/xz, I vote for USE_XZ. I disagree that USE_LZMA would be "more meaningful". There are a number of different compressors that use a variant of the LZMA algorithm, e.g. archivers/lzip, but they don't all use the same file format. -- Christian "naddy" Weisgerber naddy@mips.inka.de
Responsible Changed From-To: portmgr->pav Grab for exp http://people.freebsd.org/~mm/patches/xz/ports-lzma-3.patch
State Changed From-To: open->closed Committed with a change of exists -> osversion
pav 2010-06-04 08:09:20 UTC FreeBSD ports repository Modified files: Mk bsd.commands.mk bsd.port.mk devel/libtool22 Makefile distinfo devel/patch Makefile distinfo dns/dnsmasq Makefile distinfo ftp/lftp Makefile distinfo ftp/wget Makefile distinfo graphics/ImageMagick Makefile distinfo graphics/png Makefile distinfo japanese/font-ume Makefile math/mpfr Makefile distinfo misc/mc Makefile distinfo print/ghostscript8 Makefile distinfo sysutils/coreutils Makefile distinfo textproc/gnugrep Makefile distinfo Log: - Introduce a new USE_XZ knob that handles lzma/xz compressed distfiles - Convert a bunch of ports to use it PR: ports/146329 Submitted by: mm With hat: portmgr Revision Changes Path 1.5 +7 -1 ports/Mk/bsd.commands.mk 1.642 +12 -2 ports/Mk/bsd.port.mk 1.4 +2 -0 ports/devel/libtool22/Makefile 1.3 +3 -3 ports/devel/libtool22/distinfo 1.22 +1 -0 ports/devel/patch/Makefile 1.8 +3 -3 ports/devel/patch/distinfo 1.62 +3 -0 ports/dns/dnsmasq/Makefile 1.44 +3 -3 ports/dns/dnsmasq/distinfo 1.146 +2 -1 ports/ftp/lftp/Makefile 1.101 +3 -3 ports/ftp/lftp/distinfo 1.94 +2 -1 ports/ftp/wget/Makefile 1.31 +3 -3 ports/ftp/wget/distinfo 1.307 +2 -3 ports/graphics/ImageMagick/Makefile 1.148 +3 -3 ports/graphics/ImageMagick/distinfo 1.108 +1 -1 ports/graphics/png/Makefile 1.55 +3 -3 ports/graphics/png/distinfo 1.12 +2 -10 ports/japanese/font-ume/Makefile 1.18 +1 -1 ports/math/mpfr/Makefile 1.12 +3 -3 ports/math/mpfr/distinfo 1.129 +2 -1 ports/misc/mc/Makefile 1.36 +3 -3 ports/misc/mc/distinfo 1.198 +1 -0 ports/print/ghostscript8/Makefile 1.73 +3 -3 ports/print/ghostscript8/distinfo 1.28 +1 -0 ports/sysutils/coreutils/Makefile 1.9 +3 -3 ports/sysutils/coreutils/distinfo 1.5 +2 -0 ports/textproc/gnugrep/Makefile 1.3 +3 -3 ports/textproc/gnugrep/distinfo _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"