Bug 218811 - graphics/libglapi: fails to build with bsdgrep
Summary: graphics/libglapi: fails to build with bsdgrep
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Kyle Evans
URL:
Keywords: regression
: 218858 221758 (view as bug list)
Depends on:
Blocks: 218385
  Show dependency treegraph
 
Reported: 2017-04-22 12:13 UTC by Jan Beich
Modified: 2017-08-25 01:04 UTC (History)
5 users (show)

See Also:


Attachments
Fix, applied on top of D10433 (3.84 KB, patch)
2017-04-24 20:29 UTC, Kyle Evans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer 2017-04-22 12:13:30 UTC
The first bad is base r316477 or ports r438198.

$ make
[...]
checking for RADEON... yes
checking for RADEON... yes
configure: error: LLVM component 'asmparser' not enabled in your LLVM build. Required by r600.
===>  Script "configure" failed unexpectedly.

$ llvm-config40 --components | bsdgrep -iqw asmparser; echo $?
1
$ llvm-config40 --components | gnugrep -iqw asmparser; echo $?
0

$ llvm-config40 --components
aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo asmparser asmprinter bitreader bitwriter bpf bpfasmprinter bpfcodegen bpfdesc bpfdisassembler bpfinfo codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfomsf debuginfopdb demangle engine executionengine globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo lanaiinstprinter libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata riscv riscvcodegen riscvdesc riscvinfo runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo
Comment 1 Jan Beich freebsd_committer 2017-04-22 12:31:55 UTC
Broken word matching also manifests via --color e.g.,

$ echo 'foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux foo bar qux' |
  env -i TERM=xterm bsdgrep --color bar
load: 0.68  cmd: grep 40786 [running] 13.54r 13.52u 0.00s 74% 2264k
^C
Comment 2 Kyle Evans freebsd_committer 2017-04-22 14:15:53 UTC
(In reply to Jan Beich from comment #1)

D'oh. This is a trivially broken case; easily exceeds the maximum number of line matches (MAX_LINE_MATCHES=32), then it infinitely loops because it still matches past that but it's not able to be inserted into 'matches,' causing my failure mechanisms to fail because $bad.

I'll fix this in my -w empty broken review (https://reviews.freebsd.org/D10433) version, because it's *not* straightforward to fix this on -HEAD since procline() conflates matching a line with printing context. In the current state of that patch, this will basically record 'st' and set lastmatches=0, force a break at util.c:443, then at util.c:284 I'll throw in a loop that does a procline() and printline() as long as we max out on matches.

Thanks for the report! =)
Comment 3 Kyle Evans freebsd_committer 2017-04-22 14:45:59 UTC
To clarify, I'll fix this separately after D10433, rather than trying to squeeze this in as well. I've got a fix in progress, though, to minimize delay.
Comment 4 Ed Maste freebsd_committer 2017-04-24 19:04:32 UTC
jbeich, is this broken with current HEAD src + ports (after r316477, because the port invokes bsdgrep), or only with bsdgrep installed as grep?
Comment 5 Jan Beich freebsd_committer 2017-04-24 19:16:27 UTC
(In reply to Ed Maste from comment #4)
Only bsdgrep as grep breaks build.
Comment 6 Kyle Evans freebsd_committer 2017-04-24 20:29:14 UTC
Created attachment 182056 [details]
Fix, applied on top of D10433

Attaching a patch to be applied after/on top of D10433 to address this, along with an associated test case. printline() had to be modified because it was doing the wrong thing in the case of -o and any flag specified to present file/line/offset information.
Comment 7 Jan Beich freebsd_committer 2017-04-24 21:10:11 UTC
Comment on attachment 182056 [details]
Fix, applied on top of D10433

I confirm, this fixes comment 1 but not comment 0.
Comment 8 Kyle Evans freebsd_committer 2017-04-24 21:47:09 UTC
(In reply to Jan Beich from comment #7)

Oh, you wanted the build failure fixed, too. =-)

That part is actually fixed in D10329 [1] stemming from PR 218467. Apologies for missing that.

[1] https://reviews.freebsd.org/D10329
Comment 9 Conrad Meyer freebsd_committer 2017-04-25 00:38:55 UTC
*** Bug 218858 has been marked as a duplicate of this bug. ***
Comment 10 commit-hook freebsd_committer 2017-05-02 02:32:44 UTC
A commit references this bug:

Author: emaste
Date: Tue May  2 02:32:10 UTC 2017
New revision: 317665
URL: https://svnweb.freebsd.org/changeset/base/317665

Log:
  bsdgrep: fix -w -v matching improperly with certain patterns

  -w and -v flag matching was mostly functional but had some minor
  problems:

  1. -w flag processing only allowed one iteration through pattern
     matching on a line. This was problematic if one pattern could match
     more than once, or if there were multiple patterns and the earliest/
     longest match was not the most ideal, and

  2. Previous work "fixed" things to not further process a line if the
     first iteration through patterns produced no matches. This is clearly
     wrong if we're dealing with the more restrictive -w matching.

  #2 breakage could have also occurred before recent broad rewrites, but
  it would be more arbitrary based on input patterns as to whether or not
  it actually affected things.

  Fix both of these by forcing a retry of the patterns after advancing
  just past the start of the first match if we're doing more restrictive
  -w matching and we didn't get any hits to start with. Also move -v flag
  processing outside of the loop so that we have a greater change to match
  in the more restrictive cases. This wasn't strictly wrong, but it could
  be a little more error prone.

  While here, introduce some regressions tests for this behavior and fix
  some excessive wrapping nearby that hindered readability. GNU grep
  passes these new tests.

  PR:		218467, 218811
  Submitted by:	Kyle Evans <kevans91 at ksu.edu>
  Reviewed by:	cem, ngie
  Differential Revision:	https://reviews.freebsd.org/D10329

Changes:
  head/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  head/usr.bin/grep/util.c
Comment 11 commit-hook freebsd_committer 2017-05-20 03:52:34 UTC
A commit references this bug:

Author: emaste
Date: Sat May 20 03:51:32 UTC 2017
New revision: 318571
URL: https://svnweb.freebsd.org/changeset/base/318571

Log:
  bsdgrep: emit more than MAX_LINE_MATCHES per line

  We should not set an arbitrary cap on the number of matches on a line,
  and in any case MAX_LINE_MATCHES of 32 is much too low.  Instead, if we
  match more than MAX_LINE_MATCHES, keep processing and matching from the
  last match until all are found.

  For the regression test, we produce 4096 matches (larger than we expect
  we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096
  lines of output with the -o flag.

  We'll also make sure that every distinct line is getting its own line
  number to detect line metadata not being printed as appropriate along
  the way.

  PR:		218811
  Submitted by:	Kyle Evans <kevans91@ksu.edu>
  Reported by:	jbeich
  Reviewed by:	cem
  Differential Revision:	https://reviews.freebsd.org/D10577

Changes:
  head/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  head/usr.bin/grep/util.c
Comment 12 commit-hook freebsd_committer 2017-08-16 17:42:55 UTC
A commit references this bug:

Author: kevans
Date: Wed Aug 16 17:42:39 UTC 2017
New revision: 322583
URL: https://svnweb.freebsd.org/changeset/base/322583

Log:
  MFC r317665: bsdgrep: fix -w -v matching improperly with certain patterns

  -w and -v flag matching was mostly functional but had some minor
  problems:

  1. -w flag processing only allowed one iteration through pattern
     matching on a line. This was problematic if one pattern could match
     more than once, or if there were multiple patterns and the earliest/
     longest match was not the most ideal, and

  2. Previous work "fixed" things to not further process a line if the
     first iteration through patterns produced no matches. This is clearly
     wrong if we're dealing with the more restrictive -w matching.

  #2 breakage could have also occurred before recent broad rewrites, but
  it would be more arbitrary based on input patterns as to whether or not
  it actually affected things.

  Fix both of these by forcing a retry of the patterns after advancing
  just past the start of the first match if we're doing more restrictive
  -w matching and we didn't get any hits to start with. Also move -v flag
  processing outside of the loop so that we have a greater change to match
  in the more restrictive cases. This wasn't strictly wrong, but it could
  be a little more error prone.

  While here, introduce some regressions tests for this behavior and fix
  some excessive wrapping nearby that hindered readability. GNU grep
  passes these new tests.

  PR:		218467, 218811
  Approved by:	emaste (mentor, blanket MFC)

Changes:
_U  stable/11/
  stable/11/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  stable/11/usr.bin/grep/util.c
Comment 13 commit-hook freebsd_committer 2017-08-17 04:26:33 UTC
A commit references this bug:

Author: kevans
Date: Thu Aug 17 04:26:04 UTC 2017
New revision: 322609
URL: https://svnweb.freebsd.org/changeset/base/322609

Log:
  MFC r318571: bsdgrep: emit more than MAX_LINE_MATCHES per line

  We should not set an arbitrary cap on the number of matches on a line,
  and in any case MAX_LINE_MATCHES of 32 is much too low.  Instead, if we
  match more than MAX_LINE_MATCHES, keep processing and matching from the
  last match until all are found.

  For the regression test, we produce 4096 matches (larger than we expect
  we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096
  lines of output with the -o flag.

  We'll also make sure that every distinct line is getting its own line
  number to detect line metadata not being printed as appropriate along
  the way.

  PR:		218811
  Approved by:	emaste (mentor, blanket MFC)

Changes:
_U  stable/11/
  stable/11/contrib/netbsd-tests/usr.bin/grep/t_grep.sh
  stable/11/usr.bin/grep/util.c
Comment 14 Ed Maste freebsd_committer 2017-08-17 16:01:50 UTC
Kyle, is this now resolved?
Comment 15 Jan Beich freebsd_committer 2017-08-25 01:04:38 UTC
*** Bug 221758 has been marked as a duplicate of this bug. ***