lex : How to override YY_BUF_SIZE

Question

According to the manual YY_BUF_SIZE is 16K and we need to override it. However, the manual does not specify how to override it, nor could I find any command line option for this. Can someone please indicate how to change this. In the generated source YY_BUF_SIZE is defined as follows:

#ifndef YY_BUF_SIZE
#define YY_BUF_SIZE 16384
#endif

so there may be a way to override it before this.

Answer 1

In your own code, simply #define YY_BUF_SIZE to whatever value you want. As long as you compile your code so the compiler sees your definition first, the #ifndef guard will prevent the default value from being set.

Answer 2

Using -DYY_BUF_SIZE=99999 (for whatever number, of course) on the command line is the only way to completely override the value, it seems to me.

The built-in flex skeleton/preamble precedes any %{...%} code given in the first (definitions) section of the .l source, and the skeleton is where the #ifndef YY_BUF_SIZE is. Therefore, to override within the .l file, it's best (to avoid possible warnings, etc.) to do

#undef YY_BUF_SIZE
#define YY_BUF_SIZE 99999

The potential pitfall of doing the override this way is that the skeleton also has the line

#define YY_STATE_BUF_SIZE   ((YY_BUF_SIZE + 2) * sizeof(yy_state_type))

and therefore that value will not be affected by the redefinition. In my case that did not matter, but it's something to be aware of.

Unfortunately there is no way, as far as I know, to insert code before the built-in preamble. If there is, I'dbe grateful to learn.