Bug 186861 - [PATCH] bsdgrep(1): xzgrep(1) sometimes doesn't find matches it should find
Summary: [PATCH] bsdgrep(1): xzgrep(1) sometimes doesn't find matches it should find
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.0-RELEASE
Hardware: Any Any
: Normal Affects Some People
Assignee: Xin LI
URL:
Keywords:
: 194016 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-02-18 09:50 UTC by Antoine Brodin
Modified: 2015-02-04 00:46 UTC (History)
9 users (show)

See Also:


Attachments
patch to fix xzgrep (2.37 KB, patch)
2014-09-29 20:56 UTC, Stefan Ehmann
no flags Details | Diff
improved patch (2.68 KB, patch)
2014-09-30 19:49 UTC, Stefan Ehmann
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Antoine Brodin freebsd_committer 2014-02-18 09:50:00 UTC
xzgrep(1) sometimes doesn't find matches it should find

How-To-Repeat: % xzcat py27-virtualenv-1.11.2_1.txz | grep pkg_resources
Binary file (standard input) matches
% xzgrep pkg_resources py27-virtualenv-1.11.2_1.txz
%
Comment 1 Stefan Ehmann 2014-09-26 16:31:08 UTC
I can reproduce that bug on 10-STABLE.

xzgrep only works on the first 32K (uncompressed) of the input file.

$ seq 10000 | xz > seq.xz
$ xzgrep -c . seq.xz
6775

Using regular grep works as expected:
$ xzcat seq.xz | grep -c .
10000

zgrep/bz2grep also work.
Comment 3 Mark Felder freebsd_committer 2014-09-29 14:28:52 UTC
*** Bug 194016 has been marked as a duplicate of this bug. ***
Comment 4 Stefan Ehmann 2014-09-29 20:56:34 UTC
Created attachment 147810 [details]
patch to fix xzgrep

Here's an attempt to fix xzgrep for files > 32KB.

Seems to work basically, but not much testing done yet.

Testing/Review welcome.
Comment 5 Stefan Ehmann 2014-09-30 19:49:37 UTC
Created attachment 147858 [details]
improved patch

improved version of the patch:
* laction must be initialized to LZMA_RUN for each new file
* allocate lin_buf on heap (avoids wasting 32KB if we're not using xz)
Comment 6 commit-hook freebsd_committer 2015-01-21 01:12:27 UTC
A commit references this bug:

Author: delphij
Date: Wed Jan 21 01:11:38 UTC 2015
New revision: 277463
URL: https://svnweb.freebsd.org/changeset/base/277463

Log:
  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861
  MFC after:	2 weeks

Changes:
  head/usr.bin/grep/file.c
Comment 7 Xin LI freebsd_committer 2015-01-21 01:13:29 UTC
(In reply to Stefan Ehmann from comment #5)
I have committed your original patch plus setting laction = LZMA_RUN.

Thanks for your submission!
Comment 8 commit-hook freebsd_committer 2015-02-04 00:45:59 UTC
A commit references this bug:

Author: delphij
Date: Wed Feb  4 00:45:03 UTC 2015
New revision: 278175
URL: https://svnweb.freebsd.org/changeset/base/278175

Log:
  MFC r277463:

  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861

Changes:
_U  stable/10/
  stable/10/usr.bin/grep/file.c
Comment 9 commit-hook freebsd_committer 2015-02-04 00:46:02 UTC
A commit references this bug:

Author: delphij
Date: Wed Feb  4 00:45:29 UTC 2015
New revision: 278176
URL: https://svnweb.freebsd.org/changeset/base/278176

Log:
  MFC r277463:

  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861

Changes:
_U  stable/9/usr.bin/grep/
  stable/9/usr.bin/grep/file.c
Comment 10 Xin LI freebsd_committer 2015-02-04 00:46:44 UTC
Fix applied, thanks for your submission!