The maximal number of open files in awk(1) is limited by FOPEN_MAX constant defined for awk in usr.bin/awk/Makefile: CFLAGS+= -DHAS_ISBLANK -I. -I${AWKSRC} -DFOPEN_MAX=64 This might be low for today needs and today hardware allows much higher values. NetBSD has fixed their version of awk making the array of files dynamically allocated http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=37205 It would be nice to do the same in FreeBSD. Fix: See the attached patch adopted from NetBSD (PR/37205: Aleksey Cheusov: nawk: a number of open files is limited to very small constant. Make the array of files dynamically allocated.). Patch attached with submission follows: How-To-Repeat: kopusha:~% awk 'BEGIN {for (i=1; i <= 2000; ++i){print "test" > ("/tmp/file_" i)}}' awk: /tmp/file_62 makes too many open files source line number 1
Warner is checking out some awk changes from NetBSD.
batch change: For bugs that match the following - Status Is In progress AND - Untouched since 2018-01-01. AND - Affects Base System OR Documentation DO: Reset to open status. Note: I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f39dd6a9784467f0db5886012b3f4b13899be6b8 commit f39dd6a9784467f0db5886012b3f4b13899be6b8 Merge: 7cd22ac43418 746b7396bb3e Author: Warner Losh <imp@FreeBSD.org> AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh <imp@FreeBSD.org> CommitDate: 2021-07-08 01:25:43 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub <to.my.trociny@gmail.com> PR: 143363,143365,143368,143369,143373,143375,214782 Sponsored by: Netflix contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=666abb0888d277e82c6468851e015798e9a7629f commit 666abb0888d277e82c6468851e015798e9a7629f Author: Warner Losh <imp@FreeBSD.org> AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh <imp@FreeBSD.org> CommitDate: 2021-07-10 17:07:26 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub <to.my.trociny@gmail.com> PR: 143363, 143365, 143368, 143369, 143373, 143375, 214783 Sponsored by: Netflix (cherry picked from commit f39dd6a9784467f0db5886012b3f4b13899be6b8) contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-)