How to add additional arguments to yylex (in bison/flex)?

Question

I am trying to create a reentrant parser using flex and bison. I want to add a parameter to save some state, but I failed to add it to yylex().

Here is the example, it is not expected to compile, just shows the generated code.

foo.l

%option reentrant
%option bison-bridge
%option header-file="foo.tab.h"
%{
#include "foo.tab.h"
%}
%%
"{" { return "{"; }
")" { return '}'; }
%%

foo.y

%define api.pure full
%define parse.error verbose
%parse-param {void *scanner}
%parse-param {int *pint}
%lex-param {void *scanner}
%lex-param {int *pint}
%token '(' ')'
%%
foo : '(' | ')' ;
%%

run with:

bison -d -b foo foo.y
flex foo.l
gcc -E lex.yy.c | less

We can see int yylex (YYSTYPE * yylval_param , yyscan_t yyscanner) {...} So pint is gone. But I think I have specified it at foo.y. So what I need to do more to make yylex accept pint?

Environment: Gentoo Linux stable with Bison-3.0.4 and Flex 2.5.39

Bison directives (eg. %lex-param) have no influence on flex. How could they? Flex doesn't even know the name of the bison file. — rici
Anyway, see if stackoverflow.com/questions/34418381/… helps. If not, I'll try to write something up later. — rici
@rici , I am not a native English speaker, and I misunderstood some documents. I though %lex-param automatically add additional args in flex. — OstCollector
bison and flex are not as well-documented as one might like; even native English speakers have trouble. But they don't violate that laws of physics. Flex produces the yyflex function, with a defined parameter list. bison produces code which calls yyflex, providing an actual argument lust. They are separate tools; they don't see each other's files. But obviously the declaration and the call must agree. So the source files for the two tools need to make sure that both tools generate compatible outputs. — rici
Turns out that it is easier to pass custom data with yyextra (rather than modifying yylex signature) as explained in the answer to this question and in flex docs. — gudok

xsnpdngv xsnpdngv · Accepted Answer · 2016-09-27T18:51:49

%lex-param says for bison to invoke yylex with extra parameters, but does not say anything to flex.

The default definition of the yylex() function can be changed by defining the YY_DECL macro in your foo.l file's definition part. In order to have only the int *pint as argument, it looks like this:

#define YY_DECL int yylex(int *pint)

If yylval_param and yyscanner are also needed, then:

#define YY_DECL int yylex(YYSTYPE * yylval_param, yyscan_t yyscanner, int *pint)

How to add additional arguments to yylex (in bison/flex)?

1 Answers