Bug 253209 - grep -v -f some-empty-file -- does the wrong thing
Summary: grep -v -f some-empty-file -- does the wrong thing
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 13.0-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: Kyle Evans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-02-03 11:49 UTC by Helge Oldach
Modified: 2021-02-11 15:26 UTC (History)
4 users (show)

See Also:
kevans: mfc-stable13+
kevans: mfc-stable12+
kevans: mfc-stable11-


Attachments
git(1) diff against base (4.55 KB, patch)
2021-02-04 21:39 UTC, Kyle Evans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Helge Oldach 2021-02-03 11:49:40 UTC
# >a
# echo blubb | fgrep -f a
# echo blubb | fgrep -v -f a
#

Actually -v should invert the match however if the file is empty it seems it's not doing the right thing.

For comparison (some random RedHat):

# uname -a
Linux rh.local 3.10.0-862.3.2.el7.x86_64 #1 SMP Tue May 15 18:22:15 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
# >a
# echo blubb | fgrep -f a
# echo blubb | fgrep -v -f a
blubb
# 

I stumbled over this while debugging a non-working script. I seem to remember that it was working fine a while ago on stable/12.2.

I suspect related to base r352691 - maybe the removal of the invert logic near the end of util.c.
Comment 1 Kyle Evans freebsd_committer freebsd_triage 2021-02-04 21:39:07 UTC
Created attachment 222170 [details]
git(1) diff against base

Looks like this dates back to the 2011 update, though I certainly didn't help matters any... basically, we short-circuit back in main() far before we get to any matching stuff, but we cannot short-circuit because we may still need to output.

The attached patch fixes the semantics of -w and -x in the presence of both null patterns and empty pattern files, to make sure that we don't miss anything. I will commit it ~tonight, most likely.
Comment 2 commit-hook freebsd_committer freebsd_triage 2021-02-05 03:00:49 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f823c6dc730b0dd08b54a53be1d8fd587eee7021

commit f823c6dc730b0dd08b54a53be1d8fd587eee7021
Author:     Kyle Evans <kevans@FreeBSD.org>
AuthorDate: 2021-02-04 21:26:45 +0000
Commit:     Kyle Evans <kevans@FreeBSD.org>
CommitDate: 2021-02-05 02:59:42 +0000

    grep: fix null pattern and empty pattern file behavior

    The null pattern semantics were terrible because I tried to match gnugrep,
    but I got it wrong.  Let's unwind that:

    - The null pattern should match every line if neither -w nor -x.
    - The null pattern should match empty lines if -x.
    - The null pattern should not match any lines if -w.

    The first two will stop processing (shortcut) even if additional patterns
    are specified. In any other case, we will continue processing other
    patterns.  If no other patterns are specified beside a null pattern, then
    we match if neither -w nor -x or set and do not match if either of those
    are specified.

    The justification for -w is that it should match on a whole word, but the
    null pattern deos not have a whole word to match on.

    Empty pattern files should never match anything, and more importantly, -v
    should cause everything to be written.

    PR:             253209
    MFC-after:      4 days

 contrib/netbsd-tests/usr.bin/grep/t_grep.sh | 22 +++++++++++++++---
 usr.bin/grep/grep.c                         | 11 ---------
 usr.bin/grep/util.c                         | 35 +++++++++++++----------------
 3 files changed, 35 insertions(+), 33 deletions(-)
Comment 3 commit-hook freebsd_committer freebsd_triage 2021-02-11 02:50:45 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=574d0dfae5011a766aa967f1d1675ddf7b535936

commit 574d0dfae5011a766aa967f1d1675ddf7b535936
Author:     Kyle Evans <kevans@FreeBSD.org>
AuthorDate: 2021-02-04 21:26:45 +0000
Commit:     Kyle Evans <kevans@FreeBSD.org>
CommitDate: 2021-02-11 02:49:44 +0000

    grep: fix null pattern and empty pattern file behavior

    The null pattern semantics were terrible because I tried to match gnugrep,
    but I got it wrong.  Let's unwind that:

    - The null pattern should match every line if neither -w nor -x.
    - The null pattern should match empty lines if -x.
    - The null pattern should not match any lines if -w.

    The first two will stop processing (shortcut) even if additional patterns
    are specified. In any other case, we will continue processing other
    patterns.  If no other patterns are specified beside a null pattern, then
    we match if neither -w nor -x or set and do not match if either of those
    are specified.

    The justification for -w is that it should match on a whole word, but the
    null pattern deos not have a whole word to match on.

    Empty pattern files should never match anything, and more importantly, -v
    should cause everything to be written.

    PR:             253209
    (cherry picked from commit f823c6dc730b0dd08b54a53be1d8fd587eee7021)

 contrib/netbsd-tests/usr.bin/grep/t_grep.sh | 22 +++++++++++++++---
 usr.bin/grep/grep.c                         | 11 ---------
 usr.bin/grep/util.c                         | 35 +++++++++++++----------------
 3 files changed, 35 insertions(+), 33 deletions(-)
Comment 4 commit-hook freebsd_committer freebsd_triage 2021-02-11 15:12:52 UTC
A commit in branch releng/13.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f28ccd8d95954134cec93db482fb0bd4ce93f36e

commit f28ccd8d95954134cec93db482fb0bd4ce93f36e
Author:     Kyle Evans <kevans@FreeBSD.org>
AuthorDate: 2021-02-11 15:10:44 +0000
Commit:     Kyle Evans <kevans@FreeBSD.org>
CommitDate: 2021-02-11 15:12:10 +0000

    grep: fix null pattern and empty pattern file behavior

    The null pattern semantics were terrible because I tried to match gnugrep,
    but I got it wrong.  Let's unwind that:

    - The null pattern should match every line if neither -w nor -x.
    - The null pattern should match empty lines if -x.
    - The null pattern should not match any lines if -w.

    The first two will stop processing (shortcut) even if additional patterns
    are specified. In any other case, we will continue processing other
    patterns.  If no other patterns are specified beside a null pattern, then
    we match if neither -w nor -x or set and do not match if either of those
    are specified.

    The justification for -w is that it should match on a whole word, but the
    null pattern does not have a whole word to match on.

    Empty pattern files should never match anything, and more importantly, -v
    should cause everything to be written.

    PR:             253209
    Approved by:    re (gjb)

    (cherry picked from commit f823c6dc730b0dd08b54a53be1d8fd587eee7021)
    (cherry picked from commit 574d0dfae5011a766aa967f1d1675ddf7b535936)

 contrib/netbsd-tests/usr.bin/grep/t_grep.sh | 22 +++++++++++++++---
 usr.bin/grep/grep.c                         | 11 ---------
 usr.bin/grep/util.c                         | 35 +++++++++++++----------------
 3 files changed, 35 insertions(+), 33 deletions(-)
Comment 5 commit-hook freebsd_committer freebsd_triage 2021-02-11 15:21:56 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=92ddb6090b0c5091a64855a514676a452c10750c

commit 92ddb6090b0c5091a64855a514676a452c10750c
Author:     Kyle Evans <kevans@FreeBSD.org>
AuthorDate: 2021-02-04 21:26:45 +0000
Commit:     Kyle Evans <kevans@FreeBSD.org>
CommitDate: 2021-02-11 15:19:26 +0000

    grep: fix null pattern and empty pattern file behavior

    The null pattern semantics were terrible because I tried to match gnugrep,
    but I got it wrong.  Let's unwind that:

    - The null pattern should match every line if neither -w nor -x.
    - The null pattern should match empty lines if -x.
    - The null pattern should not match any lines if -w.

    The first two will stop processing (shortcut) even if additional patterns
    are specified. In any other case, we will continue processing other
    patterns.  If no other patterns are specified beside a null pattern, then
    we match if neither -w nor -x or set and do not match if either of those
    are specified.

    The justification for -w is that it should match on a whole word, but the
    null pattern deos not have a whole word to match on.

    Empty pattern files should never match anything, and more importantly, -v
    should cause everything to be written.

    PR:             253209

    (cherry picked from commit f823c6dc730b0dd08b54a53be1d8fd587eee7021)

 contrib/netbsd-tests/usr.bin/grep/t_grep.sh | 22 +++++++++++++++---
 usr.bin/grep/grep.c                         | 11 ---------
 usr.bin/grep/util.c                         | 35 +++++++++++++----------------
 3 files changed, 35 insertions(+), 33 deletions(-)
Comment 6 Kyle Evans freebsd_committer freebsd_triage 2021-02-11 15:26:14 UTC
This should be in -BETA2. Thanks for the report!