Bug 186861

Summary: [PATCH] bsdgrep(1): xzgrep(1) sometimes doesn't find matches it should find
Product: Base System Reporter: Antoine Brodin <antoine>
Component: binAssignee: Xin LI <delphij>
Status: Closed FIXED    
Severity: Affects Some People CC: bdrewery, delphij, emaste, feld, gabor, gavin, pfg, pi, shoesoft
Priority: Normal    
Version: 10.0-RELEASE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
patch to fix xzgrep
none
improved patch none

Description Antoine Brodin freebsd_committer freebsd_triage 2014-02-18 09:50:00 UTC 
xzgrep(1) sometimes doesn't find matches it should find

How-To-Repeat: % xzcat py27-virtualenv-1.11.2_1.txz | grep pkg_resources
Binary file (standard input) matches
% xzgrep pkg_resources py27-virtualenv-1.11.2_1.txz
%
Comment 1 Stefan Ehmann 2014-09-26 16:31:08 UTC 
I can reproduce that bug on 10-STABLE.

xzgrep only works on the first 32K (uncompressed) of the input file.

$ seq 10000 | xz > seq.xz
$ xzgrep -c . seq.xz
6775

Using regular grep works as expected:
$ xzcat seq.xz | grep -c .
10000

zgrep/bz2grep also work.
Comment 3 Mark Felder freebsd_committer freebsd_triage 2014-09-29 14:28:52 UTC 
*** Bug 194016 has been marked as a duplicate of this bug. ***
Comment 4 Stefan Ehmann 2014-09-29 20:56:34 UTC 
Created attachment 147810 [details]
patch to fix xzgrep

Here's an attempt to fix xzgrep for files > 32KB.

Seems to work basically, but not much testing done yet.

Testing/Review welcome.
Comment 5 Stefan Ehmann 2014-09-30 19:49:37 UTC 
Created attachment 147858 [details]
improved patch

improved version of the patch:
* laction must be initialized to LZMA_RUN for each new file
* allocate lin_buf on heap (avoids wasting 32KB if we're not using xz)
Comment 6 commit-hook freebsd_committer freebsd_triage 2015-01-21 01:12:27 UTC 
A commit references this bug:

Author: delphij
Date: Wed Jan 21 01:11:38 UTC 2015
New revision: 277463
URL: https://svnweb.freebsd.org/changeset/base/277463

Log:
  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861
  MFC after:	2 weeks

Changes:
  head/usr.bin/grep/file.c
Comment 7 Xin LI freebsd_committer freebsd_triage 2015-01-21 01:13:29 UTC 
(In reply to Stefan Ehmann from comment #5)
I have committed your original patch plus setting laction = LZMA_RUN.

Thanks for your submission!
Comment 8 commit-hook freebsd_committer freebsd_triage 2015-02-04 00:45:59 UTC 
A commit references this bug:

Author: delphij
Date: Wed Feb  4 00:45:03 UTC 2015
New revision: 278175
URL: https://svnweb.freebsd.org/changeset/base/278175

Log:
  MFC r277463:

  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861

Changes:
_U  stable/10/
  stable/10/usr.bin/grep/file.c
Comment 9 commit-hook freebsd_committer freebsd_triage 2015-02-04 00:46:02 UTC 
A commit references this bug:

Author: delphij
Date: Wed Feb  4 00:45:29 UTC 2015
New revision: 278176
URL: https://svnweb.freebsd.org/changeset/base/278176

Log:
  MFC r277463:

  Fix xz handling for files larger than 32K.

  Submitted by:	Stefan Ehmann <shoesoft gmx net>
  PR:		bin/186861

Changes:
_U  stable/9/usr.bin/grep/
  stable/9/usr.bin/grep/file.c
Comment 10 Xin LI freebsd_committer freebsd_triage 2015-02-04 00:46:44 UTC 
Fix applied, thanks for your submission!