How can I parse a C string (char *) with flex/bison?

8
votes

In my programming project I want to parse command line attributes using flex/bison. My program is called like this:

./prog -a "(1, 2, 3)(4, 5)(6, 7, 8)" filename

Is it possible to parse this string using flex/bison without writing it to a file and parsing that file?

5
cbisonflex-lexer
I would think that writing a simple state machine would be easier and cleaner than using Flex or Bison for this. – James McNellis
If you think you need flex and bison, how complex is this grammar anyway? And I have to agree with James: for just parsing comma-separated lists of integers with optional whitespace and parentheses, C would be best. – Mike DeSimone
I only used flex and bison together so far. Having a closer look it makes more sense to only use flex. – Philipp Riegger

5 Answers

3
votes

I think you can achieve something like that (I did a similar thing) by using fmemopen to create a stream from a char*and then replace that to stdin

Something like that (not sure if it's fully functional since I'm actually trying to remember available syscalls but it would be something similar to this)

char* args = "(1,2,3)(4,5)(6,7,8)"
FILE *newstdin = fmemopen (args, strlen (args), "r");
FILE *oldstdin = fdup(stdin);

stdin = newstdin;

// do parsing

stdin = oldstdin;
2
votes

Here is a complete flex example.

%%

<<EOF>> return 0;

.   return 1;

%%

int yywrap()
{
    return (1);
}

int main(int argc, const char* const argv[])
{
    YY_BUFFER_STATE bufferState = yy_scan_string("abcdef");

    // This is a flex source. For yacc/bison use yyparse() here ...    
    int token;
    do {
        token = yylex();
    } while (token != 0);

    // Do not forget to tell flex to clean up after itself. Lest
    // ye leak memory.
    yy_delete_buffer(bufferState);

    return (EXIT_SUCCESS);
}
0
votes

another example. this one redefines the YY_INPUT macro:

%{
int myinput (char *buf, int buflen);
char *string;
int offset;
#define YY_INPUT(buf, result, buflen) (result = myinput(buf, buflen));
%}
%%

[0-9]+             {printf("a number! %s\n", yytext);}

.                  ;
%%

int main () {
    string = "(1, 2, 3)(4, 5)(6, 7, 8)";
    yylex();
}

int myinput (char *buf, int buflen) {
    int i;
    for (i = 0; i < buflen; i++) {
        buf[i] = string[offset + i];
        if (!buf[i]) {
            break;
        }
    }
    offset += i;
    return i;
}
-1
votes

The answer is "Yes". See the O'Reilly publication called "lex & yacc", 2nd Edition by Doug Brown, John Levine, Tony Mason. Refer to Chapter 6, the section "Input from Strings".

I also just noticed that there are some good instructions in the section "Input from Strings", Chapter 5 of "flex and bison", by John Levine. Look out for routines yy_scan_bytes(char *bytes, int len), yy_scan_string("string"), and yy_scan_buffer(char *base, yy_size_t size). I have not scanned from strings myself, but will be trying it soon.