Bug 20172

Summary: yacc(1) byacc 1.9 fails to generate $default transitions for non-terminals
Product: Base System Reporter: bertho <bertho>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   

Description bertho 2000-07-25 12:50:08 UTC
Transitions on declared, but unused tokens are not included into
state transitions. You normally would get $default transitions,
but byacc insists on specifying all tokens. This leads to wrong
results because a syntax-error is generated on a perfectly legal
input.

Version of byacc:
yysccsid[] = \"@(#)yaccpar     1.9 (Berkeley) 02/21/93\";"

The bug was also sent to the maintainer as mentioned in the source's
README file. However, it is promised in that file that no rapid response
is to be expected.

Fix: 

You have to define a rule that captures all non-terminal tokens (i.e. all
tokens in %token, %left and %right declarations that are otherwise
unused). This can be tricky if such a rule has grammatical side-effects.

A real fix requires a patch in byacc (but I am not familiar with its
internals). OTOH, GNU bison does not suffer from this bug and can
be used as an alternative.
How-To-Repeat: Try this source, once without the tNL rule, and once with. A syntax
error is generated if the rule is not implemented. When the rule is
there, then the proper result is printed.

To compile (use attached source below):
$ byacc byacc-bug.y
$ gcc -o byacc-bug y.tab.c
$ ./byacc-bug

---- file byacc-bug.y ----
%{
#include <stdio.h>
#include <stdlib.h>
%}

%token tTOK tNUM tNL
%left '+'

%%
lines	: line
	| lines line
	;

line	: tTOK xpr ',' xpr {
		if(yychar == tNL)
			printf("Success: Got tNL\n");
	}
 /* Declare NON-terminal as used */
 /*	| tNL	*/
	;

xpr	: xpr '+' xpr
	| tNUM
	;
%%

int yylex(void)
{
	static int tok[] = {tTOK, tNUM, ',', tNUM, tNL, 0};
	static int idx = 0;
#define NTOK	(sizeof(tok)/sizeof(tok[0]))

	if(idx < NTOK)
		return tok[idx++];
	return 0;
}

void yyerror(char *s)
{
	printf("yyerror: %s\n", s);
	exit(1);
}

int main(void)
{
	return yyparse();
}
Comment 1 bertho 2000-07-27 09:48:31 UTC
Two comments:

1) The example should use a slightly different action for one rule to
make it more clear what happens. The yyclearin should eliminate the
lookahead and then all should continue as planned. However, byacc will
generate a syntax error on the input. Here is the rule as intended:
line: tTOK xpr ',' xpr {
	if(yychar == tNL) {
		printf("Success: Got tNL\n");
		yyclearin;
	}
    }


2) After some email exchange with the original author it has become
apparent that he does not seem to consider the problem a bug. This means
that it is likely that byacc will be unusable for all heavy yacc-parsers
that manipulate lookahead tokens in an above fassion.

Greetings Bertho
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2011-03-22 02:38:35 UTC
State Changed
From-To: open->closed

yacc author does not consider this a bug