Bug 146329

Summary: [patch] add xz support to bsd.port.mk
Product: Ports & Packages Reporter: Martin Matuska <mm>
Component: Individual Port(s)Assignee: Pav Lucistnik <pav>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
file.diff none

Description Martin Matuska freebsd_committer freebsd_triage 2010-05-05 14:40:01 UTC
Many ports offer nowadays LZMA compressed distfiles (.tar.xz) that
give (much) better compression than bzip2.

The attached patch updates bsd.port.mk and bsd.commands.mk to
support the USE_XZ flag for xz compressed files.

http://tukaani.org/xz/
Comment 1 Edwin Groothuis freebsd_committer freebsd_triage 2010-05-05 14:40:11 UTC
Responsible Changed
From-To: freebsd-ports-bugs->portmgr

bsd.port.mk is portmgr territory (via the GNATS Auto Assign Tool)
Comment 2 Florent Thoumie 2010-05-11 19:33:52 UTC
How many ports are we talking about here?

I see you just added some lzma support to base system but I don't know
whether it's compatible with archivers/xz. If it is, it might be worth
changing the patch to use it.

-- 
Florent Thoumie
flz@FreeBSD.org
FreeBSD Committer
Comment 3 Martin Matuska freebsd_committer freebsd_triage 2010-05-11 20:05:32 UTC
Yes, the patch should actually detect if it is available in the base
system and if not use the port.

We are talking about many ports but not a killing number.

I can give examples of ports distributing .xz or .lzma files:
GNU is adopting this widely: grep, patch, autoconf, automake, gawk,
libtool, wget, .....
Others: ImageMagick, ghostscript, libpng, ....

The main benefits for compressed source code are:
a) it decompresses up twice as fast than bz2 (but always faster)
b) it compresses much better than bz2 (I yield results of 30% and better
in my tests) (but always better)
Comment 4 Florent Thoumie 2010-05-11 20:14:27 UTC
On Tue, May 11, 2010 at 8:05 PM, Martin Matuska <mm@freebsd.org> wrote:
> Yes, the patch should actually detect if it is available in the base
> system and if not use the port.
>
> We are talking about many ports but not a killing number.
>
> I can give examples of ports distributing .xz or .lzma files:
> GNU is adopting this widely: grep, patch, autoconf, automake, gawk,
> libtool, wget, .....
> Others: ImageMagick, ghostscript, libpng, ....
>
> The main benefits for compressed source code are:
> a) it decompresses up twice as fast than bz2 (but always faster)
> b) it compresses much better than bz2 (I yield results of 30% and better
> in my tests) (but always better)

Beside updating the patch to use base system tools if available, could
you follow-up with a patch to update a few ports to use USE_XZ?

-- 
Florent Thoumie
flz@FreeBSD.org
FreeBSD Committer
Comment 5 Martin Matuska freebsd_committer freebsd_triage 2010-05-12 14:14:30 UTC
I will start with ImageMagick right away and I would recommend
ghostscript as well.

DÅa 11. 5. 2010 21:14, Florent Thoumie wrote / napísal(a):
> On Tue, May 11, 2010 at 8:05 PM, Martin Matuska <mm@freebsd.org> wrote:
>   
>> Yes, the patch should actually detect if it is available in the base
>> system and if not use the port.
>>
>> We are talking about many ports but not a killing number.
>>
>> I can give examples of ports distributing .xz or .lzma files:
>> GNU is adopting this widely: grep, patch, autoconf, automake, gawk,
>> libtool, wget, .....
>> Others: ImageMagick, ghostscript, libpng, ....
>>
>> The main benefits for compressed source code are:
>> a) it decompresses up twice as fast than bz2 (but always faster)
>> b) it compresses much better than bz2 (I yield results of 30% and better
>> in my tests) (but always better)
>>     
> Beside updating the patch to use base system tools if available, could
> you follow-up with a patch to update a few ports to use USE_XZ?
>
>
Comment 7 Florent Thoumie 2010-05-12 21:53:17 UTC
On Wed, May 12, 2010 at 9:39 PM, Martin Matuska <mm@freebsd.org> wrote:
> http://people.freebsd.org/~mm/patches/xz/ports-lzma.patch

Just mentioning that the patch contains the following changes that I
discussed with Martin over IRC:

- change USE_XZ to USE_LZMA because it'll be more meaningful.
- change default EXTRACT_SUFX to .tar.lzma as it seems more common than .tar.xz.

-- 
Florent Thoumie
flz@FreeBSD.org
FreeBSD Committer
Comment 8 Martin Matuska freebsd_committer freebsd_triage 2010-05-22 12:53:37 UTC
Here are the patches, but som ports got updated since I wrote the patches:

http://people.freebsd.org/~mm/patches/xz/ports-lzma.patch
http://people.freebsd.org/~mm/patches/xz/ports-lzma-2.patch

The main question is:
USE_LZMA or USE_XZ?

Some ports use .tar.xz, some .tar.lzma (both are extractable with xz)
Comment 9 Christian Weisgerber freebsd_committer freebsd_triage 2010-05-22 15:16:56 UTC
Martin Matuska:

> The main question is:
> USE_LZMA or USE_XZ?
> 
> Some ports use .tar.xz, some .tar.lzma (both are extractable with xz)

Those are different container formats.  (The container format
provides a file(1) signature, integrity checks, etc.)

Lasse Collin started the project under the name LZMA Utils and used
a provisional container format; those are the .tar.lzma file you
see.  When he had finished designing the permanent file format (see
http://tukaani.org/xz/format.html), he also changed the project
name to XZ Utils and the file suffix to .xz.  xz(1) can still extract
the old .lzma files, but they should go away eventually.

With this in mind, and speaking as the (ex-)maintainer of archivers/xz,
I vote for USE_XZ.

I disagree that USE_LZMA would be "more meaningful".  There are a
number of different compressors that use a variant of the LZMA
algorithm, e.g. archivers/lzip, but they don't all use the same
file format.

-- 
Christian "naddy" Weisgerber                          naddy@mips.inka.de
Comment 10 Pav Lucistnik freebsd_committer freebsd_triage 2010-05-31 15:11:06 UTC
Responsible Changed
From-To: portmgr->pav

Grab for exp 
http://people.freebsd.org/~mm/patches/xz/ports-lzma-3.patch
Comment 11 Pav Lucistnik freebsd_committer freebsd_triage 2010-06-04 09:04:46 UTC
State Changed
From-To: open->closed

Committed with a change of exists -> osversion
Comment 12 dfilter service freebsd_committer freebsd_triage 2010-06-04 09:09:29 UTC
pav         2010-06-04 08:09:20 UTC

  FreeBSD ports repository

  Modified files:
    Mk                   bsd.commands.mk bsd.port.mk 
    devel/libtool22      Makefile distinfo 
    devel/patch          Makefile distinfo 
    dns/dnsmasq          Makefile distinfo 
    ftp/lftp             Makefile distinfo 
    ftp/wget             Makefile distinfo 
    graphics/ImageMagick Makefile distinfo 
    graphics/png         Makefile distinfo 
    japanese/font-ume    Makefile 
    math/mpfr            Makefile distinfo 
    misc/mc              Makefile distinfo 
    print/ghostscript8   Makefile distinfo 
    sysutils/coreutils   Makefile distinfo 
    textproc/gnugrep     Makefile distinfo 
  Log:
  - Introduce a new USE_XZ knob that handles lzma/xz compressed distfiles
  - Convert a bunch of ports to use it
  
  PR:             ports/146329
  Submitted by:   mm
  With hat:       portmgr
  
  Revision  Changes    Path
  1.5       +7 -1      ports/Mk/bsd.commands.mk
  1.642     +12 -2     ports/Mk/bsd.port.mk
  1.4       +2 -0      ports/devel/libtool22/Makefile
  1.3       +3 -3      ports/devel/libtool22/distinfo
  1.22      +1 -0      ports/devel/patch/Makefile
  1.8       +3 -3      ports/devel/patch/distinfo
  1.62      +3 -0      ports/dns/dnsmasq/Makefile
  1.44      +3 -3      ports/dns/dnsmasq/distinfo
  1.146     +2 -1      ports/ftp/lftp/Makefile
  1.101     +3 -3      ports/ftp/lftp/distinfo
  1.94      +2 -1      ports/ftp/wget/Makefile
  1.31      +3 -3      ports/ftp/wget/distinfo
  1.307     +2 -3      ports/graphics/ImageMagick/Makefile
  1.148     +3 -3      ports/graphics/ImageMagick/distinfo
  1.108     +1 -1      ports/graphics/png/Makefile
  1.55      +3 -3      ports/graphics/png/distinfo
  1.12      +2 -10     ports/japanese/font-ume/Makefile
  1.18      +1 -1      ports/math/mpfr/Makefile
  1.12      +3 -3      ports/math/mpfr/distinfo
  1.129     +2 -1      ports/misc/mc/Makefile
  1.36      +3 -3      ports/misc/mc/distinfo
  1.198     +1 -0      ports/print/ghostscript8/Makefile
  1.73      +3 -3      ports/print/ghostscript8/distinfo
  1.28      +1 -0      ports/sysutils/coreutils/Makefile
  1.9       +3 -3      ports/sysutils/coreutils/distinfo
  1.5       +2 -0      ports/textproc/gnugrep/Makefile
  1.3       +3 -3      ports/textproc/gnugrep/distinfo
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"