Preprocess a custom text file to remove comments using boost spirit

Question

I've text which contains "equations" like :

-- This is comment
ABC:= 121-XY1/7 > 45 OR SS >= 3
    ZY2 AND -- This is another comment
    (JKL * PQR) < 75;

JKL:= PP1 OR 
      PP2/2 XOR PP3;

ZZ_1:=A-B > 0 XOR (B2 % GH == 6 AND 
   SP == 4
-- Again a comment
    NOT AX > GF < 2 OR C*AS2 >= 5);

I decided to use boost spirit to parse this text , as of now I just need to know my operands and operators.

I've referred this nice answer (thanks to sehe :) ) to write my expression grammar (relational operators not written yet)

However I cannot strip my comments with :-

     qi::phrase_parse(input.begin()
     ,input.end()
     ,p >> ';' // parser object
     ,qi::space | "--" >> *(qi::char_ - qi::eol) >> qi::eol
     ,result //expression object,  (boost::variant with boost::recursive_wrapper)
     );

because it gives several errors, some of the post says to tweak a boost header file.

So I'm using another grammar to strip off comment first using :

     qi::phrase_parse(input.begin()
     ,input.end()
     ,qi::char_ >> *qi::char_ 
     , qi::space | "--" >> *(qi::char_ - qi::eol) >> qi::eol
     ,stripped // std::string 
     );

But this gives me a text with all space and comment removed :

ABC:=121-XY1/7>45ORSS>=3ZY2AND(JKL*PQR)<75;JKL:=PP1ORPP2/2XORPP3;ZZ_1:=A-B>0XOR(B2%GH==6ANDSP==4NOTAX>GF<2ORC*AS2>=5);

So, the question is how can I strip just the comments, preserving space and newlines ?

^{Using : Boost Version 1.55.0}

I am processing a text file that I also need to strip out comments (C# not C++) and I simply do it with a Regex to zap everything after and including the comment marker. That way the structure of the file remains. I do this step before I parse my file. Whereas something like spirit is meant to carve the file up into tokens isn't it? — Peter M
@PeterM yeah, with python/perl regex it will be a matter of few lines, but I kinda interested in "compiler-like" functionality. Like correct evaluation of expressions, pin-point the errors, etc — P0W
I'm thinking you need to think like a C/C++ preprocessor rather than parser/compiler at this step. — Peter M
About your initial situation... How are you declaring your skipper type in your parser object? If the code you pasted is what you are using, you would need to use decltype or BOOST_TYPEOF. If that were your problem (and I'm taking a shot in the dark) I believe the usual solution is creating a skipper grammar (something like this). — llonesmiz

sehe sehe · Accepted Answer · 2014-02-01T19:44:42

Your exact code sample is missing. Allow me to add a sample skipper to that "boolean expression grammar" you linked to:

std::string const input = 
        "a and\n"
        "-- abacadabra\n"
        "b;";

typedef std::string::const_iterator It;

// ADDED: allow comments
qi::rule<It> skip_ws_and_comments 
    = qi::space 
    | "--" >> *(qi::char_-qi::eol) >> qi::eol
    ;

parser<It, qi::rule<It> > p;

That's all the changes required. Output:

result: (a & b)

See it Live On Coliru

Preprocess a custom text file to remove comments using boost spirit

1 Answers