Common tokens for flex and bison

Question

I have one file with declarations of my tokens declarations.h:

#define ID 257
#define NUM 258
...

In my flex code i return one of this values or symbol(for example '+', '-', '*'). And everything works.

The problem in bison file. If i write something like that: exp: ID '+' ID i'll get error, because bison doesn't know anything about ID. Adding the line %token ID will not help, because in that case i'll have compilation error(preprocessor will change ID by 257 and i'll get 257=257)

Jonathan Leffler Jonathan Leffler · Accepted Answer · 2011-12-04T15:44:06

You get Bison to create the list of tokens; your lexer uses the list generated by Bison.

bison -d grammar.y
# Generates grammar.tab.c and grammar.tab.h

Your lexer then uses grammar.tab.h:

$ cat grammar.y
%token ID
%%
program:    /* Nothing */
    |       program ID
    ;
%%
$ cat lexer.l
%{
#include "grammar.tab.h"
%}
%%
[a-zA-Z][A-Za-z_0-9]+   { return ID; }
[ \t\n]                 { /* Nothing */ }
.                       { return *yytext; }
%%
$ bison -d grammar.y
$ flex lexer.l
$ gcc -o testgrammar grammar.tab.c lex.yy.c -ly -lfl
$ ./testgrammar
id est
quod erat demonstrandum
$

Bison 2.4.3 on MacOS X 10.7.2 generates the token numbers as an enum, not as a series of #define values - to get the token names into the symbol table for debuggers (a very good idea!).

Common tokens for flex and bison

1 Answers