How to unlex using Flex(The Fast Lexical Analyzer)?

Question

Is there any way to put a token back into the input stream using Flex? I imagine some function like yyunlex().

Answer 1

There is the macro REJECT which will put the token back to stream and continue to match other the rules as though the first matched didn't. If you just want to put some char back to stream @Kizaru's answer will suffice.

Example snippet:

%%
a     |
ab    |
abc   |
abcd  ECHO; REJECT;
.|\n  printf("xx%c", *yytext);
%%

Answer 2

You have a few options.

You can put each character for the token back onto the input stream using unput(ch) where ch is the character. This call puts ch as the next character on the input stream (next character to be considered in scanning). So you could do this if you save the string during the token match.

You might want to look into yyless(0) which will put all of the characters from the token back onto the input stream too. I never used this one though, so I'm not sure if there are any gotchas. You can specify an integer n hwich will put all but the first n characters back on the input stream.

Now, if you're going to do this often during scanning/parsing, you might want to use lex just to build tokens and place the tokens onto your own data structure for parsing. This is akin to what bison and yacc's generated yyparse() function does.