Bug 220924 - bsdgrep(1) unexpectedly stops processing command line arguments
Summary: bsdgrep(1) unexpectedly stops processing command line arguments
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.0-RELEASE
Hardware: Any Any
: --- Affects Many People
Assignee: Kyle Evans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-22 13:35 UTC by Joachim K
Modified: 2017-08-29 21:23 UTC (History)
4 users (show)

See Also:
kevans: mfc-stable11+
kevans: mfc-stable10?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joachim K 2017-07-22 13:35:51 UTC
When searching for an empty expression '' with bsdgrep(1), processing of files given on the command line stops when an empty file is encountered.

$ touch a
$ echo b > b
$ bsdgrep '' a b
$ 

The output is empty, while the expected output is a line containing 'b:b'.

This does not occur when passing the file names in the opposite order:

$ bsdgrep '' b a
b:b
$ 

Note that grep(1) (GNU grep) behaves as expected.

$ grep '' a b
b:b
$ grep '' b a
b:b
$
Comment 1 Bob Bishop 2017-07-22 15:15:45 UTC
(In reply to Joachim K from comment #0)
Per re_format(7): "A (modern) RE is one or more non-empty branches, separated by `|'." It doesn't say that basic regular expressions differ in this respect, so it looks like the empty string is not a valid RE and so there may be more than one bug here.
Comment 2 Joachim K 2017-07-22 16:00:19 UTC
As another data point, bsdgrep behaves the same with the -E or -F switches. AFAICS the manual does not specify the semantics of an empty string for -F.
Comment 3 Kyle Evans freebsd_committer freebsd_triage 2017-07-22 18:37:54 UTC
I'll poke at the specifics later, but empty strings with grep(1) triggers a match-all behavior for what I assume are POSIX reasons. Empty patterns are not valid EREs or BREs- just some grep specific behavior.

My assumption at the moment is that we're just bailing out on the first file since it's a match-all. Not at a machine to test it at the moment, though.
Comment 4 Bob Bishop 2017-07-22 18:42:57 UTC
(In reply to Kyle Evans from comment #3)
If that's POSIXly correct behaviour, then re_format(7) is in error.
Comment 5 Bob Bishop 2017-07-22 18:47:16 UTC
(In reply to rb from comment #4)
... or maybe bsdgrep(1).
Comment 6 Joachim K 2017-07-22 19:58:47 UTC
(In reply to Kyle Evans from comment #3)
My experiments seem to suggest it's really the empty file that triggers it:

$ bsdgrep '' b b a b
b:b
b:b
$
Comment 7 Kyle Evans freebsd_committer freebsd_triage 2017-07-22 20:07:19 UTC
The exact wording in POSIX 1003.1-2008's page on grep is:

> a null BRE shall match every line. 

There's similar wording for EREs further down, as well as -F. I would link directly to it, but later versions aren't great for that, but: see: http://pubs.opengroup.org/onlinepubs/9699919799/ -- this doc bug should definitely be corrected if we aren't noting it in grep(1).

I'll regain access to a machine here in a couple hours to check why the empty file is killing it. Thanks for confirming that!
Comment 8 Kyle Evans freebsd_committer freebsd_triage 2017-07-22 23:21:47 UTC
(In reply to Kyle Evans from comment #7)

Relevant review: https://reviews.freebsd.org/D11698

I've made no changes to bsdgrep(1) because it is at least lightly noted in paragraph 1, sentence 3 of the description:

> An empty expression matches every line.

This could be more emphasized and noted as being the case with all types of expressions, but I'm no good with wording in manpages. =)
Comment 9 commit-hook freebsd_committer freebsd_triage 2017-07-25 01:50:50 UTC
A commit references this bug:

Author: kevans
Date: Tue Jul 25 01:50:37 UTC 2017
New revision: 321450
URL: https://svnweb.freebsd.org/changeset/base/321450

Log:
  bsdgrep(1): Don't exit before processing every file

  Given an empty pattern (i.e. grep "" A B), bsdgrep(1) would previously exit()
  with the appropriate exit code upon encountering an empty file. Likely intended
  as an optimization, but this behavior is technically incorrect since an empty
  pattern should match every line.

  PR:		220924
  Reviewed by:	emaste, cem (earlier version), ngie
  Approved by:	emaste (mentor)
  Differential Revision:	https://reviews.freebsd.org/D11698

Changes:
  head/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  head/usr.bin/grep/util.c
Comment 10 commit-hook freebsd_committer freebsd_triage 2017-08-22 02:03:31 UTC
A commit references this bug:

Author: kevans
Date: Tue Aug 22 02:03:02 UTC 2017
New revision: 322777
URL: https://svnweb.freebsd.org/changeset/base/322777

Log:
  MFC r321450: bsdgrep(1): Don't exit before processing every file

  Given an empty pattern (i.e. grep "" A B), bsdgrep(1) would previously
  exit() with the appropriate exit code upon encountering an empty file.
  Likely intended as an optimization, but this behavior is technically
  incorrect since an empty pattern should match every line.

  PR:		220924
  Approved by:	emaste (mentor, blanket MFC)

Changes:
_U  stable/11/
  stable/11/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  stable/11/usr.bin/grep/util.c