devel/bison: core dump on NLS and colorized enabled after 3.6.4. Tested with 3.7.6, 3.7.5, 3.7.4 in ports, with or without the changes other than version informataion in Makefile. (USES+= iconv, NLS_BUILD_DEPENDS= libtextstyle>=0.21:devel/libtextstyle) lwhsu@x1c:~/tmp > cat foo.y %pure-parser lwhsu@x1c:~/tmp > bison --version bison (GNU Bison) 3.7.6 Written by Robert Corbett and Richard Stallman. Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. lwhsu@x1c:~/tmp > bison foo.y foo.y:1.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [[1m% lwhsu@x1c:~/tmp > bison --color=no foo.y foo.y:1.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated] 1 | %pure-parser | ^~~~~~~~~~~~ | %define api.pure foo.y:2.1: error: unexpected end of file foo.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother] lwhsu@x1c:~/tmp > bison --version bison (GNU Bison) 3.6.4 Written by Robert Corbett and Richard Stallman. Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. lwhsu@x1c:~/tmp > bison foo.y foo.y:1.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated] 1 | %pure-parser | ^~~~~~~~~~~~ | %define api.pure foo.y:2.1: error: unexpected end of file foo.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother] lwhsu@x1c:~/tmp > lldb /usr/local/bin/bison (lldb) target create "/usr/local/bin/bison" Current executable set to '/usr/local/bin/bison' (x86_64). (lldb) settings set -- target.run-args foo.y (lldb) run Process 52566 launching Process 52566 launched: '/usr/local/bin/bison' (x86_64) foo.y:1.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [[31msignal SIGSEGV: invalid address (fault address: 0x0) frame #0: 0x0000000000000000 error: Bad address (lldb) bt * thread #1, name = 'bison', stop reason = signal SIGSEGV: invalid address (fault address: 0x0) * frame #0: 0x0000000000000000 frame #1: 0x000000080084b42a libncursesw.so.9`delay_output_sp(sp=0x0000000000000000, ms=<unavailable>) at lib_tputs.c:104:6 frame #2: 0x000000080084bcb0 libncursesw.so.9`tputs_sp [inlined] delay_output(ms=<unavailable>) at lib_tputs.c:116:12 frame #3: 0x000000080084bca1 libncursesw.so.9`tputs_sp(sp=<unavailable>, string="", affcnt=1, outc=<unavailable>) at lib_tputs.c:432 frame #4: 0x000000080084be2b libncursesw.so.9`tputs(string="982f397d0005c5334f28943700000000", affcnt=1, outc=(libtextstyle.so.0`___lldb_unnamed_symbol150$$libtextstyle.so.0)) at lib_tputs.c:454:12 frame #5: 0x000000080030b81c libtextstyle.so.0`___lldb_unnamed_symbol149$$libtextstyle.so.0 + 92 frame #6: 0x000000080030c25f libtextstyle.so.0`___lldb_unnamed_symbol152$$libtextstyle.so.0 + 943 frame #7: 0x000000080030bb2f libtextstyle.so.0`___lldb_unnamed_symbol151$$libtextstyle.so.0 + 543 frame #8: 0x0000000800309790 libtextstyle.so.0`___lldb_unnamed_symbol125$$libtextstyle.so.0 + 32 frame #9: 0x0000000000223258 bison`___lldb_unnamed_symbol42$$bison + 968 frame #10: 0x0000000000222e6f bison`___lldb_unnamed_symbol41$$bison + 127 frame #11: 0x0000000000223632 bison`___lldb_unnamed_symbol46$$bison + 98 frame #12: 0x00000000002438bf bison`___lldb_unnamed_symbol241$$bison + 5615 frame #13: 0x000000000024b579 bison`___lldb_unnamed_symbol296$$bison + 57 frame #14: 0x000000000023644e bison`___lldb_unnamed_symbol194$$bison + 318 frame #15: 0x000000000021edc0 bison`___lldb_unnamed_symbol1003$$bison + 256 (lldb)
I cannot reproduce this in a 11.4-R poudriere jail. % cat foo.y %define api.pure % cat foo2.y %pure-parser # lldb /usr/local/bin/bison (lldb) target create "/usr/local/bin/bison" Current executable set to '/usr/local/bin/bison' (x86_64). (lldb) settings set -- target.run-args foo.y (lldb) run Process 64970 launching Process 64970 launched: '/usr/local/bin/bison' (x86_64) foo.y:2.1: error: unexpected end of file Process 64970 exited with status = 1 (0x00000001) (lldb) ^D lldb /usr/local/bin/bison (lldb) target create "/usr/local/bin/bison" Current executable set to '/usr/local/bin/bison' (x86_64). (lldb) settings set -- target.run-args foo2.y (lldb) run Process 65464 launching Process 65464 launched: '/usr/local/bin/bison' (x86_64) foo2.y:1.1-12: warning: deprecated directive: '%pure-parser', use '%define api.pure' [-Wdeprecated] 1 | %pure-parser | ^~~~~~~~~~~~ | %define api.pure foo2.y:2.1: error: unexpected end of file foo2.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother] Process 65464 exited with status = 1 (0x00000001) (lldb) ^D
(In reply to Po-Chuan Hsieh from comment #1) Hmm, I was on a 13.0-R bare metal machine. I've tested the other 2, now I have 2 have the same issue but 1 is fine. I'll check more about that's the difference in those environments.
Per Alexander Mishin's comment here, https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254211 I have built Bison without NLS support. It seems to work now. It was crashing for me when building nfdump from source.
Based on the test last weekend, it seems to be an ncurses issue. bison with NLS will use ncurses via libtextstyle. It works fine with base ncurses on 11.4-R [1], 12.2-R [2] and 14-CURRENT [3]. It failed on 13-R with base ncurses (6.2-20200215) [4]. On this box, it works again after switching to ncurses port (6.2-20210403). [1] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/11.4 ncurses 5.9 20140222 [2] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/12.2 ncurses 5.9 20140222 [3] https://cgit.freebsd.org/src/log/contrib/ncurses ncurses 6.2-20210220 [4] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/13.0 ncurses 6.2-20200215
Given the backtrace and the commit message, I wonder if this is fixed by https://github.com/mirror/ncurses/commit/d30f99439fcc8d4bb4c38e5c4afb4f6555fc6ad4#diff-c9b60e2c13cb0715a1d4779df4615221bec5a65da9ec68aa00b8abcb5b5d4559 specifically (I can't seem to find the actual upstream bug report nor the patch for just that issue, and upstream is stuck in the past with its development practices, so that's all the information I can find about that one-line fix). If so perhaps it's worth of a 13.0R-p4?
I ran the command on freefall and it works without error. I guess a possible fix is to MFC ncurses 6.2-20210220 (7a65641922f404b84e9a249d48593de84d8e8d17) to 13.0-R.
I also get this on a recent install. FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr 9 04:24:09 UTC 2021 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 The ncurses package from branches/2021Q3 doesn't help (ncurses-6.2.20210313_1), not even when I LD_PRELOAD it (which has to be done, as the bison specifically wants libncursesw.so.9 which is not in the package). Using yacc instead works, but is different enough to break other things.
(In reply to Jessica Clarke from comment #5) Jessica you're the best! Rebuilding the /usr/src ncurses with that one line change does stop it from crashing. Thank you so much :). Can we please get that change included in 13.0R-p4? It's basically just this: diff --git a/contrib/ncurses/ncurses/tinfo/lib_tputs.c b/contrib/ncurses/ncurses/tinfo/lib_tputs.c index bfde26efd980..f6e8a3a7da0e 100644 --- a/contrib/ncurses/ncurses/tinfo/lib_tputs.c +++ b/contrib/ncurses/ncurses/tinfo/lib_tputs.c @@ -419,7 +419,7 @@ NCURSES_SP_NAME(tputs) (NCURSES_SP_DCLx */ if (trailpad > 0 && (always_delay || normal_delay)) - delay_output(trailpad / 10); + NCURSES_SP_NAME(delay_output) (NCURSES_SP_ARGx trailpad / 10); #endif /* BSD_TPUTS */ SetOutCh(my_outch);
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b2da1032397e3339fbeebcd57b1f179e1d8a2e19 commit b2da1032397e3339fbeebcd57b1f179e1d8a2e19 Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2021-08-11 16:54:29 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2021-08-11 17:08:23 +0000 ncurses: Apply a tputs() fix from patch 20210403 From the (substantially larger) upstream commit: + call delay_output_sp to handle BSD-style padding when tputs_sp is called, whether directly or internally, to ensure that the SCREEN pointer is passed correctly (reports by Henric Jungheim, Juraj Lutter). This fixes bison segfaults observed when colourized output is enabled. Thanks to jrtc27@ for identifying the upstream fix. PR: 256731 MFC after: 3 days contrib/ncurses/ncurses/tinfo/lib_tputs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=9c15ec46bfb6558c42a668afeef1a4418dcd970d commit 9c15ec46bfb6558c42a668afeef1a4418dcd970d Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2021-08-11 16:54:29 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2021-08-14 01:13:16 +0000 ncurses: Apply a tputs() fix from patch 20210403 From the (substantially larger) upstream commit: + call delay_output_sp to handle BSD-style padding when tputs_sp is called, whether directly or internally, to ensure that the SCREEN pointer is passed correctly (reports by Henric Jungheim, Juraj Lutter). This fixes bison segfaults observed when colourized output is enabled. Thanks to jrtc27@ for identifying the upstream fix. PR: 256731 MFC after: 3 days (cherry picked from commit b2da1032397e3339fbeebcd57b1f179e1d8a2e19) contrib/ncurses/ncurses/tinfo/lib_tputs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Since I had an older 13-, and didn't search bugzilla before starting, I re-debugged this whole issue. There were a couple of workarounds available: `stty speed 38400` being the weirdest. That works because the matching of tty baud rates and output speeds is broken in ncurses -- too many signed short <-> unsigned short <-> int conversions happen -- and so for speeds above 32767 no output delays are done. Now they still aren't done, but they won't dereference a NULL pointer, if they were. PR 260016 works around this problem from another side: libtextstyle shouldn't call `tputs()` with arbitrary user strings. The user string it generates starts with a digit, which BSD ncurses interprets as a delay, which causes the call to `delay_output()` -- it also eats part of the output, which wasn't the point. I've blogged some more investigations at https://euroquis.nl/freebsd/2021/11/24/bison.html (also about fixing things in terminal emulators to properly accept the URLs bison is outputting).
(In reply to Adriaan de Groot from comment #11) Wow. Thanks for the excellent blog post. Given the amount of time that's been sunk into this, I'm thinking we need to release an EN with the ncurses patch...
(In reply to Mark Johnston from comment #12) What branches was this merged to? For those that dont/didnt need the merge, please set mfc-stableX to - Thanks!