Bug 214783
| Summary: | [patch] awk(1) POSIX character classes never match backslash | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | martin | ||||
| Component: | bin | Assignee: | Warner Losh <imp> | ||||
| Status: | Closed FIXED | ||||||
| Severity: | Affects Only Me | CC: | imp, pstef, sh+freebsd-bugzilla | ||||
| Priority: | --- | Keywords: | patch | ||||
| Version: | 10.1-RELEASE | ||||||
| Hardware: | Any | ||||||
| OS: | Any | ||||||
| Attachments: |
|
||||||
Warner is collecting awk patches. Hi, Is there an Update to this? We've just spend a good day with 3 people to figure out why FreeBSD awk is behaving different than all other BSD awks, just to find an unapplied patch from years ago that would solve the problem. Can we have this one applied, please? Best Regards, Stefan I didn't apply this patch to the tree because it was one of several patches I staged up, only to find that one of them was causing problems... Not sure it was this one or not, and got pulled away on other things. A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f39dd6a9784467f0db5886012b3f4b13899be6b8 commit f39dd6a9784467f0db5886012b3f4b13899be6b8 Merge: 7cd22ac43418 746b7396bb3e Author: Warner Losh <imp@FreeBSD.org> AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh <imp@FreeBSD.org> CommitDate: 2021-07-08 01:25:43 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub <to.my.trociny@gmail.com> PR: 143363,143365,143368,143369,143373,143375,214782 Sponsored by: Netflix contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-) and yes, the commit had the wrong bug number in it :( (In reply to Warner Losh from comment #4) I've reopened this because the update has missed an important part of my patch: - if (!adjbuf((char **) &buf, &bufsz, bp-buf+1, 100, (char **) &bp, "relex2")) + if (!adjbuf((char **) &buf, &bufsz, bp-buf+2, 100, (char **) &bp, "relex2")) This is needed to prevent heap buffer overflow, e.g. for awk '/[[:cntrl:]01234567[:graph:]]/' See also https://github.com/onetrueawk/awk/issues/121 I'll make a new PR if you prefer. (In reply to martin from comment #6) Please make a pull request upstream, and link the issue here. While I generally prefer new bugs for this situation, since you've already reopened this one, let's use it to track the issue upstream. If they merge it soon, I'll reimport. If they say they'll merge it but take a while, then I'll make a local change and drop it when it actually gets merged. Thanks for spotting this! A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=666abb0888d277e82c6468851e015798e9a7629f commit 666abb0888d277e82c6468851e015798e9a7629f Author: Warner Losh <imp@FreeBSD.org> AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh <imp@FreeBSD.org> CommitDate: 2021-07-10 17:07:26 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub <to.my.trociny@gmail.com> PR: 143363, 143365, 143368, 143369, 143373, 143375, 214783 Sponsored by: Netflix (cherry picked from commit f39dd6a9784467f0db5886012b3f4b13899be6b8) contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-) Warner, do you think we can close this now? Upstream now has a pull request to fix the heap buffer overflow: https://github.com/onetrueawk/awk/pull/132 We should pull in the change from upstream once it lands. Given 12.x timelines, I may try to pull it in soon if it isn't resolved in the next couple of days. It's still a problem. This works now, so closing since we've imported from upstream. |
Created attachment 177332 [details] Patch to escape chars in POSIX character classes The commands below should print "yes" but on FreeBSD they print nothing: echo '\' | awk '/[[:print:]]/ { print "yes" }' The attached patch fixes this (and also a theoretical problem with matching hyphen).