Bug 193499 - [tests] usr.bin/yacc/err_syntax27.error failures with the latest kyua
Summary: [tests] usr.bin/yacc/err_syntax27.error failures with the latest kyua
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-testing mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-09-09 17:28 UTC by Ngie Cooper
Modified: 2017-12-17 07:12 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ngie Cooper freebsd_committer 2014-09-09 17:28:44 UTC
From https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedBuild/testReport/usr.bin.yacc/yacc_tests/main/ :

** testing ./err_syntax27.y
...diff ./yacc/err_syntax27.error
--- ./yacc/err_syntax27.error	2014-09-09 11:03:41.000000000 +0000
+++ ./yacc/test-err_syntax27.error	2014-09-09 11:03:47.000000000 +0000
@@ -1 +0,0 @@
-YACC: e - line 3 of "./err_syntax27.y", missing '}'
...ok ./yacc/err_syntax27.output
...diff ./yacc/err_syntax27.tab.c
--- ./yacc/err_syntax27.tab.c	2014-09-09 11:03:41.000000000 +0000
+++ ./yacc/test-err_syntax27.tab.c	2014-09-09 11:03:47.000000000 +0000
@@ -1,15 +0,0 @@
-/* original parser id follows */
-/* yysccsid[] = "@(#)yaccpar	1.9 (Berkeley) 02/21/93" */
-/* (use YYMAJOR/YYMINOR for ifdefs dependent on parser version) */
-
-#define YYBYACC 1
-#define YYMAJOR 1
-#define YYMINOR 9
-#define YYCHECK "yyyymmdd"
-
-#define YYEMPTY        (-1)
-#define yyclearin      (yychar = YYEMPTY)
-#define yyerrok        (yyerrflag = 0)
-#define YYRECOVERING() (yyerrflag != 0)
-#define YYENOMEM       (-2)
-#define YYEOF          0
...ok ./yacc/err_syntax27.tab

Reported by rodrigc@
Comment 1 Ngie Cooper freebsd_committer 2014-09-09 20:39:39 UTC
This isn't an issue with 0.8 run on my VM. This might be a regression with kyua..
Comment 2 Ngie Cooper freebsd_committer 2014-09-09 20:41:04 UTC
Other info:

$ (cd /usr/tests/usr.bin/yacc/; sudo kyua test; uname -a)
yacc_tests:main  ->  passed  [1.798s]

1/1 passed (0 failed)
Committed action 1
FreeBSD fuji-current.local 11.0-CURRENT FreeBSD 11.0-CURRENT #9 r271303+aefd4c2(isilon-atf-integrate-dtrace)-dirty: Tue Sep  9 13:18:58 PDT 2014     ngie@isilon-fuji-current.local:/usr/obj/usr/src/sys/FUJI  i386
Comment 3 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2014-10-03 17:19:39 UTC
I can't imagine how this can be a Kyua bug given that 1) the test is using the plain interface, 2) the output of the test points at differences in yacc output and 3) yacc crashed.

But it seems that Garrett (?) has found a bug in yacc with valgrind so that may explain it.
Comment 4 Craig Rodrigues freebsd_committer 2014-10-03 19:38:16 UTC
When I run this test, I get a coredump in yacc,
so it is not a bug in kyua.

When I look at the coredump, I get no symbols.
I need to recompile yacc with debug symbols and provide the core.
Comment 5 Craig Rodrigues freebsd_committer 2014-10-04 09:29:56 UTC
Here's the stack trace I got:

(gdb) where
#0  reader () at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:681
#1  0x00000000004053c9 in main (argc=<value optimized out>, argv=<value optimized out>)
    at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/main.c:692
(gdb) l
676         {
677             if (c == '\0')
678                 missing_brace();
679             if (c == EOF)
680                 unexpected_EOF();
681             buf[i] = (char)c;
682         }
683
684         if (i == 0)
685             goto out;
Comment 6 Craig Rodrigues freebsd_committer 2014-10-05 07:22:39 UTC
I recompiled byacc without optimization, by doing:
make DEBUG_FLAGS="-g -O0".  Here is what I got from the backtrace.

(gdb) where
#0  0x00000000004134ed in copy_param (k=13) at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:681
#1  0x000000000040cf9d in read_declarations () at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:1359
#2  0x000000000040ce03 in reader () at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:3250
#3  0x0000000000405ed2 in main (argc=8, argv=0x7fffffffeb90) at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/main.c:692
(gdb) frame 0
#0  0x00000000004134ed in copy_param (k=13) at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:681
681             buf[i] = (char)c;

(gdb) info locals
buf = 0x8010140f0 "int regs[26] \n", '¥' <repeats 83 times>, "int regs[26] \n", '¥' <repeats 83 times>, "int re"...
c = -91
head = (param *) 0xffffffea00000000
p = (param *) 0x41241c
i = 4112144
name = -5424
type2 = 0

(gdb) frame 1
#1  0x000000000040cf9d in read_declarations () at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:1359
1359                copy_param(k);
(gdb) info locals
c = 37
k = 13

(gdb) frame 2
#2  0x000000000040ce03 in reader () at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/reader.c:3250
3250        read_declarations();
(gdb) info locals
No locals.

(gdb) frame 3
#3  0x0000000000405ed2 in main (argc=8, argv=0x7fffffffeb90) at /builds/FreeBSD_HEAD/usr.bin/yacc/../../contrib/byacc/main.c:692
692         reader();
(gdb) info locals
No locals.
(gdb) p argv[0]
$1 = 0x7fffffffedf8 "../yacc"
(gdb) p argv[1]
$2 = 0x7fffffffee00 "-v"
(gdb) p argv[2]
$3 = 0x7fffffffee03 "-d"
(gdb) p argv[3]
$4 = 0x7fffffffee06 "-p"
(gdb) p argv[4]
$5 = 0x7fffffffee09 "err_syntax27_"
(gdb) p argv[5]
$6 = 0x7fffffffee17 "-b"
(gdb) p argv[6]
$7 = 0x7fffffffee1a "test-err_syntax27"
(gdb) p argv[7]
$8 = 0x7fffffffee2c "./err_syntax27.y"
(gdb) p argv[8]
$9 = 0x0
Comment 7 Craig Rodrigues freebsd_committer 2014-10-05 08:17:01 UTC
Some more info.  If I run this testcase under valgrind, I get some
warnings.

valgrind --track-origins=yes -v yacc -v -d -p err_syntax27_ -b test-err_syntax27 ./err_syntax27.y

==57756== Memcheck, a memory error detector
==57756== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==57756== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==57756== Command: yacc -v -d -p err_syntax27_ -b test-err_syntax27 ./err_syntax27.y
==57756== 
--57756-- Valgrind options:
--57756--    --track-origins=yes
--57756--    -v
--57756-- Contents of /proc/version:
--57756-- Arch and hwcaps: AMD64, amd64-sse3-cx16-avx
--57756-- Page sizes: currently 4096, max supported 4096
--57756-- Valgrind library directory: /usr/local/lib/valgrind
--57756-- Reading syms from /libexec/ld-elf.so.1
--57756--    object doesn't have a symbol table
--57756-- Reading syms from /usr/bin/byacc
--57756-- Reading syms from /usr/local/lib/valgrind/memcheck-amd64-freebsd
--57756--    object doesn't have a symbol table
--57756--    object doesn't have a dynamic symbol table
--57756-- Scheduler: using generic scheduler lock implementation.
--57756-- Reading suppressions file: /usr/local/lib/valgrind/default.supp
==57756== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-57756-by-rodrigc-on-???
==57756== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-57756-by-rodrigc-on-???
==57756== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-57756-by-rodrigc-on-???
==57756== 
==57756== TO CONTROL THIS PROCESS USING vgdb (which you probably
==57756== don't want to do, unless you know exactly what you're doing,
==57756== or are doing some strange experiment):
==57756==   /usr/local/lib/valgrind/../../bin/vgdb --pid=57756 ...command...
==57756== 
==57756== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==57756==   /path/to/gdb yacc
==57756== and then give GDB the following command
==57756==   target remote | /usr/local/lib/valgrind/../../bin/vgdb --pid=57756
==57756== --pid is optional if only one valgrind process is running
==57756== 
--57756-- Reading syms from /usr/local/lib/valgrind/vgpreload_core-amd64-freebsd.so
--57756--    object doesn't have a symbol table
--57756-- Reading syms from /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so
--57756--    object doesn't have a symbol table
--57756-- Reading syms from /lib/libc.so.7
--57756--    object doesn't have a symbol table
--57756-- REDIR: 0x136c1e0 (strncmp) redirected to 0x1025080 (strncmp)
--57756-- REDIR: 0x1371520 (memset) redirected to 0x1025e60 (memset)
--57756-- REDIR: 0x136c220 (strlen) redirected to 0x10247b0 (strlen)
--57756-- REDIR: 0x12d8e10 (malloc) redirected to 0x10211c0 (malloc)
--57756-- REDIR: 0x136b520 (strcpy) redirected to 0x1024810 (strcpy)
--57756-- REDIR: 0x1303d50 (rindex) redirected to 0x1023fe0 (rindex)
--57756-- REDIR: 0x1371580 (memcpy) redirected to 0x1025330 (memcpy)
--57756-- REDIR: 0x136c300 (index) redirected to 0x10240a0 (index)
--57756-- REDIR: 0x12dbdb0 (calloc) redirected to 0x10230e0 (calloc)
--57756-- REDIR: 0x136b600 (strcmp) redirected to 0x1025170 (strcmp)
==57756== Conditional jump or move depends on uninitialised value(s)
==57756==    at 0x4134B6: copy_param (reader.c:675)
==57756==    by 0x40CF9C: read_declarations (reader.c:1359)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756==  Uninitialised value was created by a heap allocation
==57756==    at 0x1021293: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==57756==    by 0x410BDC: get_line (reader.c:128)
==57756==    by 0x40ECFE: nextc (reader.c:214)
==57756==    by 0x40CEC8: read_declarations (reader.c:1311)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756== 
==57756== Conditional jump or move depends on uninitialised value(s)
==57756==    at 0x4134C3: copy_param (reader.c:677)
==57756==    by 0x40CF9C: read_declarations (reader.c:1359)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756==  Uninitialised value was created by a heap allocation
==57756==    at 0x1021293: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==57756==    by 0x410BDC: get_line (reader.c:128)
==57756==    by 0x40ECFE: nextc (reader.c:214)
==57756==    by 0x40CEC8: read_declarations (reader.c:1311)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756== 
yacc: e - line 3 of "./err_syntax27.y", missing '}'
--57756-- REDIR: 0x12df240 (free) redirected to 0x10223c0 (free)
==57756== 
==57756== HEAP SUMMARY:
==57756==     in use at exit: 8,769 bytes in 9 blocks
==57756==   total heap usage: 17 allocs, 8 frees, 17,084 bytes allocated
==57756== 
==57756== Searching for pointers to 9 not-freed blocks
==57756== Checked 4,540,024 bytes
==57756== 
==57756== LEAK SUMMARY:
==57756==    definitely lost: 0 bytes in 0 blocks
==57756==    indirectly lost: 0 bytes in 0 blocks
==57756==      possibly lost: 0 bytes in 0 blocks
==57756==    still reachable: 8,769 bytes in 9 blocks
==57756==         suppressed: 0 bytes in 0 blocks
==57756== Rerun with --leak-check=full to see details of leaked memory
==57756== 
==57756== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
==57756== 
==57756== 1 errors in context 1 of 2:
==57756== Conditional jump or move depends on uninitialised value(s)
==57756==    at 0x4134C3: copy_param (reader.c:677)
==57756==    by 0x40CF9C: read_declarations (reader.c:1359)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756==  Uninitialised value was created by a heap allocation
==57756==    at 0x1021293: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==57756==    by 0x410BDC: get_line (reader.c:128)
==57756==    by 0x40ECFE: nextc (reader.c:214)
==57756==    by 0x40CEC8: read_declarations (reader.c:1311)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756== 
==57756== 
==57756== 1 errors in context 2 of 2:
==57756== Conditional jump or move depends on uninitialised value(s)
==57756==    at 0x4134B6: copy_param (reader.c:675)
==57756==    by 0x40CF9C: read_declarations (reader.c:1359)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756==  Uninitialised value was created by a heap allocation
==57756==    at 0x1021293: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==57756==    by 0x410BDC: get_line (reader.c:128)
==57756==    by 0x40ECFE: nextc (reader.c:214)
==57756==    by 0x40CEC8: read_declarations (reader.c:1311)
==57756==    by 0x40CE02: reader (reader.c:3250)
==57756==    by 0x405ED1: main (main.c:692)
==57756== 
==57756== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Comment 8 Craig Rodrigues freebsd_committer 2014-10-05 20:41:17 UTC
I managed to get rid of the valgrind errors with this patch to byacc:

Index: contrib/byacc/reader.c
===================================================================
--- contrib/byacc/reader.c      (revision 272333)
+++ contrib/byacc/reader.c      (working copy)
@@ -134,6 +134,7 @@
     for (;;)
     {
        line[i] = (char)c;
+       line[i + 1] = 0;
        if (c == '\n')
            break;
        if (++i >= linesize)
Comment 9 dickey 2014-10-06 00:36:35 UTC
I made a different change on Thursday, using calloc to allocate the
line data.  That's overkill.  At the same time, I had promised to
make %parse-param work with %destructor, and had started that.  So
I've been working on this (and aside from retesting, static analysis)
and "done".

However, I took a look at this change (undoing the calloc), and
it did not (for whatever reason) entirely fix the valgrind warnings.
In my current code, I got this (with btyacc_demo.y):

==11640== 2 errors in context 1 of 2:
==11640== Invalid write of size 1
==11640==    at 0x4095DB: get_line (reader.c:137)
==11640==    by 0x4097B6: nextc (reader.c:261)
==11640==    by 0x40B04A: copy_destructor (reader.c:2703)  
==11640==    by 0x40BDEA: read_declarations (reader.c:1564)
==11640==    by 0x40DF64: reader (reader.c:3433)
==11640==    by 0x404BBE: main (main.c:692)
==11640==  Address 0x518e7b5 is 0 bytes after a block of size 101 alloc'd
==11640==    at 0x4C244E8: malloc (vg_replace_malloc.c:236)
==11640==    by 0x4095A9: get_line (reader.c:128)
==11640==    by 0x409664: next_inline (reader.c:215)
==11640==    by 0x409759: nextc (reader.c:258)   
==11640==    by 0x40B28A: read_declarations (reader.c:1502)
==11640==    by 0x40DF64: reader (reader.c:3433)  
==11640==    by 0x404BBE: main (main.c:692)

I did use valgrind to spot the problem in err_syntax27.y, so we're
talking about the same original report.

Anyway, since I have a working fix, I'll continue with that.

(thanks for the feedback)
Comment 10 dickey 2014-10-06 10:28:19 UTC
I finished testing my snapshot from yesterday, found no new issues
so I uploaded that (20141005).  Craig reminded me that the calloc
change does not address realloc's - since valgrind is happy with the
current change, I deferred rewriting get_line to the next set of
changes.
Comment 11 commit-hook freebsd_committer 2014-10-06 14:44:02 UTC
A commit references this bug:

Author: rodrigc
Date: Mon Oct  6 14:43:03 UTC 2014
New revision: 272649
URL: https://svnweb.freebsd.org/changeset/base/272649

Log:
  MFV:
   use calloc in get_line() when allocating line to ensure it is fully initialized,
   fixes a later uninitialized value in copy_param() (FreeBSD #193499).

  PR: 193499
  Submitted by: Thomas E. Dickey  <tom@invisible-island.net>

Changes:
  head/contrib/byacc/CHANGES
  head/contrib/byacc/defs.h
  head/contrib/byacc/reader.c
Comment 12 commit-hook freebsd_committer 2014-10-11 19:28:49 UTC
A commit references this bug:

Author: rodrigc
Date: Sat Oct 11 19:28:24 UTC 2014
New revision: 272954
URL: https://svnweb.freebsd.org/changeset/base/272954

Log:
  Merge: r272649
   use calloc in get_line() when allocating line to ensure it is fully initialized,
   fixes a later uninitialized value in copy_param() (FreeBSD #193499).

  PR: 193499
  Submitted by: Thomas E. Dickey  <tom@invisible-island.net>

Changes:
_U  stable/10/
  stable/10/contrib/byacc/CHANGES
  stable/10/contrib/byacc/defs.h
  stable/10/contrib/byacc/reader.c
Comment 13 commit-hook freebsd_committer 2014-10-11 19:38:52 UTC
A commit references this bug:

Author: rodrigc
Date: Sat Oct 11 19:38:31 UTC 2014
New revision: 272957
URL: https://svnweb.freebsd.org/changeset/base/272957

Log:
  Merge: r272769

  Import byacc 20141006

  PR: 193499

Changes:
_U  stable/10/
  stable/10/contrib/byacc/CHANGES
  stable/10/contrib/byacc/MANIFEST
  stable/10/contrib/byacc/VERSION
  stable/10/contrib/byacc/configure
  stable/10/contrib/byacc/configure.in
  stable/10/contrib/byacc/main.c
  stable/10/contrib/byacc/package/byacc.spec
  stable/10/contrib/byacc/package/debian/changelog
  stable/10/contrib/byacc/package/debian/source/format
  stable/10/contrib/byacc/package/mingw-byacc.spec
  stable/10/contrib/byacc/package/pkgsrc/Makefile
  stable/10/contrib/byacc/reader.c
Comment 14 Craig Rodrigues freebsd_committer 2014-11-24 09:03:32 UTC
Fixed
Comment 15 Craig Rodrigues freebsd_committer 2014-11-24 09:04:15 UTC
Fix didn't make it into 10.1-RELEASE, but is in stable/10 branch
Comment 16 vali gholami 2017-12-17 07:12:07 UTC
MARKED AS SPAM