Bug 181263 - grep(1) crashes at memchr/__sfvwrite when using colors
Summary: grep(1) crashes at memchr/__sfvwrite when using colors
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: Kyle Evans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-08-13 14:40 UTC by kenorb
Modified: 2017-08-29 21:20 UTC (History)
5 users (show)

See Also:
kevans: mfc-stable11+
kevans: mfc-stable10?


Attachments
Proposed patch (1.10 KB, text/plain)
2014-10-04 14:16 UTC, David CARLIER
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description kenorb 2013-08-13 14:40:00 UTC
BSD grep is crashing on different behavior when using --colors parameter.
Check the below examples.

Related posts:
http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026502.html
http://stackoverflow.com/questions/17247142/grep-f-on-os-x-produces-segfault/18210372

Fix: 

Don't use colors e.g.
--color=never
How-To-Repeat: 1.
$ echo 'abcd*' | grep --color=auto -e ".*" -e a
Segmentation fault: 11

2.
$ echo "fojeiwuroiuwet\nljfajsljkfabcdddjlfkajlkj\nabcaaa\nzzzabc\n" | grep --color -f grepfile
Segmentation fault: 11

3.
echo 'abcd*' > grepfile
echo 'abc$' >> grepfile
echo '^abc' >> grepfile
echo "fojeiwuroiuwet\nljfajsljkfabcdddjlfkajlkj\nabcaaa\nzzzabc\n" | grep --color=auto -f grepfile
Comment 1 kenorb 2013-08-13 14:46:58 UTC
Backtraces:

Unix:

Program received signal SIGSEGV, Segmentation fault.
  0x0000000801007ff2 in memchr (s=0x61167a, c=10, n=18446744073707490297)
at /usr/src/lib/libc/string/memchr.c:48
  48                              if (*p++ == (unsigned char)c)
  (gdb) bt
  #0  0x0000000801007ff2 in memchr (s=0x61167a, c=10,
n=18446744073707490297) at /usr/src/lib/libc/string/memchr.c:48
  #1  0x0000000801007b03 in __sfvwrite (fp=0x801247770, uio=0x7fffffffd8f0)
at /usr/src/lib/libc/stdio/fvwrite.c:170
  #2  0x0000000801007698 in fwrite (buf=0x608c03,
size=18446744073709551606, count=1, fp=0x801247770)
      at /usr/src/lib/libc/stdio/fwrite.c:95
  #3  0x0000000000405498 in printline (line=0x7fffffffdb70, sep=58,
matches=0x7fffffffd990, m=9)
      at /usr/src/usr.bin/grep/util.c:500
  #4  0x0000000000404f51 in procline (l=0x7fffffffdb70, nottext=0) at
/usr/src/usr.bin/grep/util.c:381

OSX:
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib               0x00007fff8a8da1b3 memchr + 15
1   libsystem_c.dylib               0x00007fff8a90b45a __sfvwrite + 578
2   libsystem_c.dylib               0x00007fff8a90b861 fwrite + 114
3   grep                            0x000000010a4a3138 0x10a4a0000 + 12600
4   grep                            0x000000010a4a2988 0x10a4a0000 + 10632
5   grep                            0x000000010a4a1c28 0x10a4a0000 + 7208
6   libdyld.dylib                   0x00007fff8daf77e1 start + 1
Comment 2 David CARLIER 2014-10-04 14:16:03 UTC
Created attachment 147970 [details]
Proposed patch
Comment 3 Jake Zimmerman 2015-12-04 07:17:52 UTC
Has there been any progress on this? I just ran into this bug using exactly the same steps to reproduce as below.
Comment 4 David Carlier 2015-12-04 09:18:38 UTC
Is the proposed patch above helpful?
regards.
Comment 5 Kyle Evans freebsd_committer 2017-01-21 05:33:54 UTC
This crash should be amongst those fixed by the patch in bug #195763
Comment 6 commit-hook freebsd_committer 2017-04-03 23:17:34 UTC
A commit references this bug:

Author: emaste
Date: Mon Apr  3 23:16:51 UTC 2017
New revision: 316477
URL: https://svnweb.freebsd.org/changeset/base/316477

Log:
  bsdgrep: fix matching behaviour

  - Set REG_NOTBOL if we've already matched beginning of line and we're
    examining later parts

  - For each pattern we examine, apply it to the remaining bits of the
    line rather than (potentially) smaller subsets

  - Check for REG_NOSUB after we've looked at all patterns initially
    matching the line

  - Keep track of the last match we made to later determine if we're
    simply not matching any longer or if we need to proceed another byte
    because we hit a zero-length match

  - Match the earliest and longest bit of each line before moving the
    beginning of what we match to further in the line, past the end of the
    longest match; this generally matches how gnugrep(1) seems to behave,
    and seems like pretty good behavior to me

  - Finally, bail out of printing any matches if we were set to print all
    (empty pattern) but -o (output matches) was set

  PR:		195763, 180990, 197555, 197531, 181263, 209116
  Submitted by:	"Kyle Evans" <kevans91@ksu.edu>
  Reviewed by:	cem
  MFC after:	1 month
  Relnotes:	Yes
  Differential Revision:	https://reviews.freebsd.org/D10104

Changes:
  head/usr.bin/grep/util.c
Comment 7 commit-hook freebsd_committer 2017-04-05 18:42:15 UTC
A commit references this bug:

Author: emaste
Date: Wed Apr  5 18:41:47 UTC 2017
New revision: 316536
URL: https://svnweb.freebsd.org/changeset/base/316536

Log:
  bsdgrep: create additional tests for coverage on recent fixes

  Create additional tests to cover regressions that were discovered by
  PRs linked to reviews D10098, D10102, and D10104.

  It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base
  system currently pass all of these tests, and gnugrep(1) not quite being
  up to snuff was also noted in at least one of the PRs.

  PR:		175314 202022 195763 180990 197555 197531 181263 209116
  Submitted by:	Kyle Evans <kevans91@ksu.edu>
  Reviewed by:	cem, ngie, emaste
  MFC after:	1 month
  Differential Revision:	https://reviews.freebsd.org/D10112

Changes:
  head/contrib/netbsd-tests/usr.bin/grep/d_color_a.in
  head/contrib/netbsd-tests/usr.bin/grep/d_color_a.out
  head/contrib/netbsd-tests/usr.bin/grep/d_color_b.in
  head/contrib/netbsd-tests/usr.bin/grep/d_color_b.out
  head/contrib/netbsd-tests/usr.bin/grep/d_color_c.out
  head/contrib/netbsd-tests/usr.bin/grep/d_escmap.in
  head/contrib/netbsd-tests/usr.bin/grep/d_f_file_empty.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_a.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_a.out
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_b.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_b.out
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_c.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_c.out
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_d.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_e.in
  head/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_e.out
  head/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  head/usr.bin/grep/tests/Makefile
Comment 8 commit-hook freebsd_committer 2017-08-16 00:12:36 UTC
A commit references this bug:

Author: kevans
Date: Wed Aug 16 00:12:25 UTC 2017
New revision: 322555
URL: https://svnweb.freebsd.org/changeset/base/322555

Log:
  bsdgrep: Fix matching behavior and add regression tests

  MFC r316477: bsdgrep: fix matching behaviour

  - Set REG_NOTBOL if we've already matched beginning of line and we're
    examining later parts

  - For each pattern we examine, apply it to the remaining bits of the
    line rather than (potentially) smaller subsets

  - Check for REG_NOSUB after we've looked at all patterns initially
    matching the line

  - Keep track of the last match we made to later determine if we're
    simply not matching any longer or if we need to proceed another byte
    because we hit a zero-length match

  - Match the earliest and longest bit of each line before moving the
    beginning of what we match to further in the line, past the end of the
    longest match; this generally matches how gnugrep(1) seems to behave,
    and seems like pretty good behavior to me

  - Finally, bail out of printing any matches if we were set to print all
    (empty pattern) but -o (output matches) was set

  MFC r316489: bsdgrep: Initialize vars to avoid a false positive GCC warning

  MFC r316491: bsdgrep: revert color changes from r316477

  r316477 changed the color output to match exactly the in-tree GNU grep,
  but introduces unnecessary escape sequences.

  MFC r316536: bsdgrep: create additional tests for coverage on recent fixes

  Create additional tests to cover regressions that were discovered by
  PRs linked to reviews D10098, D10102, and D10104.

  It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base
  system currently pass all of these tests, and gnugrep(1) not quite being
  up to snuff was also noted in at least one of the PRs.

  MFC r317052: bsdgrep: fix zero-length matches without the -o flag

  r316477 broke zero-length matches when not using the -o flag, by
  skipping over them entirely.

  Add a regression test so that it doesn't break again in the future.

  PR:		175314, 180990, 181263, 195763, 197531, 197555, 202022, 209116
  Approved by:	emaste (mentor, blanket MFC)
  Relnotes:	yes

Changes:
_U  stable/11/
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_color_a.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_color_a.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_color_b.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_color_b.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_color_c.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_escmap.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_f_file_empty.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_a.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_a.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_b.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_b.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_c.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_c.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_d.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_e.in
  stable/11/contrib/netbsd-tests/usr.bin/grep/d_oflag_zerolen_e.out
  stable/11/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  stable/11/usr.bin/grep/tests/Makefile
  stable/11/usr.bin/grep/util.c