Bug 256731

Comment 1 Po-Chuan Hsieh 2021-06-20 15:52:58 UTC

I cannot reproduce this in a 11.4-R poudriere jail.

% cat foo.y
%define api.pure

% cat foo2.y
%pure-parser

# lldb /usr/local/bin/bison
(lldb) target create "/usr/local/bin/bison"
Current executable set to '/usr/local/bin/bison' (x86_64).
(lldb) settings set -- target.run-args foo.y
(lldb) run
Process 64970 launching
Process 64970 launched: '/usr/local/bin/bison' (x86_64)
foo.y:2.1: error: unexpected end of file
Process 64970 exited with status = 1 (0x00000001)
(lldb) ^D

 lldb /usr/local/bin/bison
(lldb) target create "/usr/local/bin/bison"
Current executable set to '/usr/local/bin/bison' (x86_64).
(lldb) settings set -- target.run-args foo2.y
(lldb) run
Process 65464 launching
Process 65464 launched: '/usr/local/bin/bison' (x86_64)
foo2.y:1.1-12: warning: deprecated directive: '%pure-parser', use '%define api.pure' [-Wdeprecated]
    1 | %pure-parser
      | ^~~~~~~~~~~~
      | %define api.pure
foo2.y:2.1: error: unexpected end of file
foo2.y: warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
Process 65464 exited with status = 1 (0x00000001)
(lldb) ^D

Comment 2 Li-Wen Hsu 2021-06-20 16:45:24 UTC

(In reply to Po-Chuan Hsieh from comment #1)
Hmm, I was on a 13.0-R bare metal machine. I've tested the other 2, now I have 2 have the same issue but 1 is fine.

I'll check more about that's the difference in those environments.

Per Alexander Mishin's comment here,
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254211

I have built Bison without NLS support. It seems to work now. It was crashing for me when building nfdump from source.

Comment 4 Po-Chuan Hsieh 2021-07-01 15:04:48 UTC

Based on the test last weekend, it seems to be an ncurses issue.
bison with NLS will use ncurses via libtextstyle.
It works fine with base ncurses on 11.4-R [1], 12.2-R [2] and 14-CURRENT [3].
It failed on 13-R with base ncurses (6.2-20200215) [4].
On this box, it works again after switching to ncurses port (6.2-20210403).

[1] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/11.4
    ncurses 5.9 20140222
[2] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/12.2
    ncurses 5.9 20140222
[3] https://cgit.freebsd.org/src/log/contrib/ncurses
    ncurses 6.2-20210220
[4] https://cgit.freebsd.org/src/log/contrib/ncurses?h=releng/13.0
    ncurses 6.2-20200215

Comment 5 Jessica Clarke 2021-07-05 17:44:21 UTC

Given the backtrace and the commit message, I wonder if this is fixed by https://github.com/mirror/ncurses/commit/d30f99439fcc8d4bb4c38e5c4afb4f6555fc6ad4#diff-c9b60e2c13cb0715a1d4779df4615221bec5a65da9ec68aa00b8abcb5b5d4559 specifically (I can't seem to find the actual upstream bug report nor the patch for just that issue, and upstream is stuck in the past with its development practices, so that's all the information I can find about that one-line fix). If so perhaps it's worth of a 13.0R-p4?

Comment 6 Po-Chuan Hsieh 2021-07-06 18:18:23 UTC

I ran the command on freefall and it works without error. I guess a possible fix is to MFC ncurses 6.2-20210220 (7a65641922f404b84e9a249d48593de84d8e8d17) to 13.0-R.

I also get this on a recent install.

FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 04:24:09 UTC 2021     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64

The ncurses package from branches/2021Q3 doesn't help (ncurses-6.2.20210313_1), not even when I LD_PRELOAD it (which has to be done, as the bison specifically wants libncursesw.so.9 which is not in the package).

Using yacc instead works, but is different enough to break other things.

(In reply to Jessica Clarke from comment #5)

Jessica you're the best! Rebuilding the /usr/src ncurses with that one line change does stop it from crashing. Thank you so much :).

Can we please get that change included in 13.0R-p4? It's basically just this:



diff --git a/contrib/ncurses/ncurses/tinfo/lib_tputs.c b/contrib/ncurses/ncurses/tinfo/lib_tputs.c
index bfde26efd980..f6e8a3a7da0e 100644
--- a/contrib/ncurses/ncurses/tinfo/lib_tputs.c
+++ b/contrib/ncurses/ncurses/tinfo/lib_tputs.c
@@ -419,7 +419,7 @@ NCURSES_SP_NAME(tputs) (NCURSES_SP_DCLx
      */
     if (trailpad > 0
        && (always_delay || normal_delay))
-       delay_output(trailpad / 10);
+       NCURSES_SP_NAME(delay_output) (NCURSES_SP_ARGx trailpad / 10);
 #endif /* BSD_TPUTS */
 
     SetOutCh(my_outch);

Comment 9 commit-hook 2021-08-11 17:10:29 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=b2da1032397e3339fbeebcd57b1f179e1d8a2e19

commit b2da1032397e3339fbeebcd57b1f179e1d8a2e19
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2021-08-11 16:54:29 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2021-08-11 17:08:23 +0000

    ncurses: Apply a tputs() fix from patch 20210403

    From the (substantially larger) upstream commit:
    + call delay_output_sp to handle BSD-style padding when tputs_sp is
      called, whether directly or internally, to ensure that the SCREEN
      pointer is passed correctly (reports by Henric Jungheim, Juraj
      Lutter).

    This fixes bison segfaults observed when colourized output is enabled.
    Thanks to jrtc27@ for identifying the upstream fix.

    PR:             256731
    MFC after:      3 days

 contrib/ncurses/ncurses/tinfo/lib_tputs.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comment 10 commit-hook 2021-08-14 01:13:26 UTC

A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=9c15ec46bfb6558c42a668afeef1a4418dcd970d

commit 9c15ec46bfb6558c42a668afeef1a4418dcd970d
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2021-08-11 16:54:29 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2021-08-14 01:13:16 +0000

    ncurses: Apply a tputs() fix from patch 20210403

    From the (substantially larger) upstream commit:
    + call delay_output_sp to handle BSD-style padding when tputs_sp is
      called, whether directly or internally, to ensure that the SCREEN
      pointer is passed correctly (reports by Henric Jungheim, Juraj
      Lutter).

    This fixes bison segfaults observed when colourized output is enabled.
    Thanks to jrtc27@ for identifying the upstream fix.

    PR:             256731
    MFC after:      3 days

    (cherry picked from commit b2da1032397e3339fbeebcd57b1f179e1d8a2e19)

 contrib/ncurses/ncurses/tinfo/lib_tputs.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comment 11 Adriaan de Groot 2021-11-24 16:56:15 UTC

Since I had an older 13-, and didn't search bugzilla before starting, I re-debugged this whole issue. There were a couple of workarounds available: `stty speed 38400` being the weirdest. That works because the matching of tty baud rates and output speeds is broken in ncurses -- too many signed short <-> unsigned short <-> int conversions happen -- and so for speeds above 32767 no output delays are done. Now they still aren't done, but they won't dereference a NULL pointer, if they were.

PR 260016 works around this problem from another side: libtextstyle shouldn't call `tputs()` with arbitrary user strings. The user string it generates starts with a digit, which BSD ncurses interprets as a delay, which causes the call to `delay_output()` -- it also eats part of the output, which wasn't the point.

I've blogged some more investigations at https://euroquis.nl/freebsd/2021/11/24/bison.html (also about fixing things in terminal emulators to properly accept the URLs bison is outputting).

Comment 12 Mark Johnston 2021-11-25 14:46:39 UTC

(In reply to Adriaan de Groot from comment #11)
Wow.  Thanks for the excellent blog post.  Given the amount of time that's been sunk into this, I'm thinking we need to release an EN with the ncurses patch...

Comment 13 Kubilay Kocak 2021-12-09 22:24:08 UTC

(In reply to Mark Johnston from comment #12)

What branches was this merged to? For those that dont/didnt need the merge, please set mfc-stableX to -

Thanks!