Bison bug when passing functions as arguments to yyparse?

Question

I am in the process of re-writing a parser in order to make it reentrant. In that spirit, I am interested in passing a couple of functions to bison and then have bison pass them to lex also. One of those functions is a callback which I use in my actions and the other one is the the function to be called by flex to get input data.

To do so, I have put this in my .y file:

%lex-param {void (*my_input)(void *, char*, int *, int)}
%parse-param {void (*my_input)(void *, char*, int *, int)}
%parse-param {void *(*my_callback)(void *, char *, int, struct YYLTYPE *, int, ...)}

Then, in my .l file I have declared:

#define YY_DECL int yylex (YYSTYPE *yylval_param, YYLTYPE *yylloc_param, yyscan_t yyscanner, void (*my_input)(void *, char*, int *, int))

The problem is that I think bison might have a bug when generating its code. When I try bo build, i get the following error:

tmp.tab.c: In function ‘yy_symbol_value_print’:
tmp.tab.c:4043: error: expected expression before ‘)’ token

If we visit that line we get this function:

static void
yy_symbol_value_print (FILE *yyoutput, int yytype, YYSTYPE const * const yyvaluep, YYLTYPE const * const yylocationp, yyscan_t scanner, void (*my_input)(void *, char*, int *, int), void *(*my_callback)(void *, char *, int, struct YYLTYPE *, int, ...))
{
    FILE *yyo = yyoutput;
    YYUSE (yyo);
    YYUSE (yylocationp);
    YYUSE (scanner);
    YYUSE (int);
    YYUSE (int);
    if (!yyvaluep)
        return;
    YYUSE (yytype);
}

The first line with YYUSE(int) is the one throwing the error. As you can see, this function somehow receives the same arguments as yyparse, then it calls a macro called YYUSE() with the arguments received. I think that since two of my arguments are functions (with their arglist, as they have to be declared if I understand correctly) bison calls YYUSE() with the last argument of each of those function prototypes... as far as I know it should be USE(my_input) and USE(my_callback)...

I have a hard time believing this is really a bug, I mean, really, nobody has tried this until now? I find it hard to believe...

The YYUSE() calls are all over the generated files, even though I dont really know what they are for... So changing by hand is not really an option... Has anybody done this successfully in the past? Is there something I am doing wrong?

rici rici · Accepted Answer · 2017-02-06T15:25:07

Bison's parameter parser is a little primitive. It expects to see something like %param {type name}, and if it finds something more complicated, it may do the wrong thing. (type can be reasonably complicated; it can include const, *, and other such modifiers. But the name needs to be the last thing in the specification.)

This is documented in the Bison manual: (emphasis added)

Directive: %parse-param {argument-declaration} …

Declare that one or more argument-declaration are additional yyparse arguments. The argument-declaration is used when declaring functions or prototypes. The last identifier in argument-declaration must be the argument name.

A similar restriction applies to tagnames in a %union directive.

You can make your program more readable both for bison and for human readers by using typedefs:

Put this in a common header file:

typedef void (*InputFunction)(void *, char*, int *, int);
typedef void *(*CallbackFunction)(void *, char *, int, struct YYLTYPE *, int, ...);

Bison file:

%lex-param {InputFunction my_input}
%parse-param {InputFunction my_input} {CallbackFunction my_callback}

Flex file:

#define YY_DECL int yylex (YYSTYPE *yylval_param, YYLTYPE *yylloc_param, yyscan_t yyscanner, InputFunction my_input)

By the way, the intent of the YY_USE macro is to mark parameters as "used", even if they are not; this avoids compiler warnings in functions which happen not to use the arguments. It could be argued that it is the programmer's responsibility to ensure that arguments are either used or marked as unused, but that's not the approach bison happened to take. Regardless, non-conforming parameter declarations such as the one you provided will fail in other interesting ways, even without YY_USE.

Bison bug when passing functions as arguments to yyparse?

1 Answers

Put this in a common header file:

Bison file:

Flex file: