There's a number of issues.
- Like the comment said, don't use using-directives; they land you in trouble
if all you want to concatenate all source strings in nestedSomething then, just wrap it all in a raw[] (or as_string[raw[...]] but that's not even necessary), e.g.
nestedSomething = !recursiveImpCpp >> qi::raw[*~char_("(){}")
>> -('(' >> nestedSomething >> ')')
>> -('{' >> nestedSomething >> '}')
>> !recursiveImpCpp >> *~char_("(){}")];
that rule is broken in the sense that it will match an empty string. This makes the grammar never end (it will match an "infinite" amount of empty nestedSomething). You will have to decide on some non-optional part. Here's a brute-force fix:
qi::raw[...] [ qi::_pass = px::size(qi::_1) > 0 ];
Even with that the grammar gets stuck in an infinite loop for non-trivial programs (try it with its own source). The following should clear things up a little:
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}");
parenthesized = '(' >> -nestedSomething >> ')';
bracketed = '{' >> -nestedSomething >> '}';
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -bracketed
>> -identifier_soup] [ qi::_pass = px::size(qi::_1) > 0 ];
But that will still not parse e.g. int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }). The reason is that main(<parenthesized>){<bracketed>} does only accept nestedSomething inside the brackets; that expressly prohibits the if-else construct...
Let's rename ifElseContent to something proper (like statement)
block = '{' >> startRule >> '}';
statement = block | singleStatement;
and use it instead of the bracketed:
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -block
>> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
General various notes
- you can simplify the includes, instead of wrapping them in regions to hide them
- you can simplify the rest too (see my demo)
- the grammar is going to be quite inefficient with all the negative look-aheads. Consider tokenizing or using a much more finegrained keyword detection scheme (using expectation points and or
distinct()[] from the Qi Repository)
Partial Demo
Applying the notes above:
Live On Coliru
#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/include/adapted.hpp>
#include <boost/optional/optional_io.hpp>
#ifdef BOOST_SPIRIT_DEBUG
namespace std {
template <typename... T, typename... V>
auto& operator<<(basic_ostream<T...>& os, vector<V...> const& v) {
os << "{";
for (auto& el : v) os << el << " ";
return os;
}
}
#endif
namespace qi = boost::spirit::qi;
/* later make this variant with all impStatementVariants -> either make this a
* vector to have sequences on all levels or make imperativeCpp derive from
* this
-> typedef variant<
recursive_wrapper<ifElseStruct>,
recursive_wrapper<switchStruct>,
recursive_wrapper<forStruct>,
recursive_wrapper<whileStruct>,
recursive_wrapper<doWhileStruct>
*
*/
struct ifElseStruct;
typedef boost::variant<ifElseStruct, std::string> someSeqNode;
struct ifElseStruct
{
std::string ifCond;
std::vector<someSeqNode> ifContent;
boost::optional<std::vector<someSeqNode>> elseContent;
friend std::ostream& operator<< (std::ostream& stream, const ifElseStruct& var) {
stream << "ifCond: " << var.ifCond << " ifContent: " << var.ifContent << std::endl << "elseContent:" << var.elseContent;
return stream;
}
};
BOOST_FUSION_ADAPT_STRUCT(ifElseStruct, ifCond, ifContent, elseContent)
//GRAMMAR for flowcontrol (branching and looping)
template<typename Iterator, typename Skipper> struct imperativeGrammar :qi::grammar<Iterator, std::vector<someSeqNode>(), Skipper>
{
imperativeGrammar() : imperativeGrammar::base_type(startRule)
{
startRule = *(recursiveImpCpp | nestedSomething); //vector<variant<ifElseStruct(), string>>
recursiveImpCpp = ifElseNormalRule.alias() /*| switchRule | whileRule | forRule ...*/;
//attr: ifElseStruct containing-> string, vector<someSeqNode>, optional<vector<someSeqNode>>
ifElseNormalRule = qi::lit("if") >> '(' >> condition >> ')' >> statement >> -("else" >> statement);
condition = *~qi::char_(")");//TODO: replace with nestedSomething rule
block = '{' >> startRule >> '}';
statement = block | singleStatement;
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}");
parenthesized = '(' >> -nestedSomething >> ')';
bracketed = '{' >> -nestedSomething >> '}';
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -block
>> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
singleStatement = !recursiveImpCpp >> qi::raw[*~qi::char_(';') >> ';'];
BOOST_SPIRIT_DEBUG_NODES((startRule)(ifElseNormalRule)(statement)(block)(nestedSomething)(identifier_soup)(parenthesized)(bracketed)(singleStatement))
}
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> startRule;
qi::rule<Iterator, ifElseStruct(), Skipper> recursiveImpCpp;
qi::rule<Iterator, ifElseStruct(), Skipper> ifElseNormalRule;
qi::rule<Iterator, std::string(), Skipper> condition;
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> block, statement;
qi::rule<Iterator, std::string(), Skipper> nestedSomething;
qi::rule<Iterator, std::string(), Skipper> singleStatement;
qi::rule<Iterator, Skipper> identifier_soup, parenthesized, bracketed;
/*qi::rule<Iterator, Skipper> forRule;
qi::rule<Iterator, Skipper> switchCaseBreakRule;
qi::rule<Iterator, Skipper> whileRule;
qi::rule<Iterator, Skipper> doWhileRule;*/
};
#include <fstream>
int main() {
//std::string const input = { std::istreambuf_iterator<char>(std::ifstream("main.cpp").rdbuf()), {} };
std::string const input = "int main() { if(true) { std::cout << \"Yes\\n\"; } else { std::cout << \"No\\n\"; } }";
using It = std::string::const_iterator;
It f(input.begin()), l(input.end());
imperativeGrammar<It, qi::space_type> p;
std::vector<someSeqNode> rep;
bool ok = phrase_parse(f, l, p, qi::space, rep);
if (ok) {
std::cout << "Parse success: " << rep << "\n";
}
else
std::cout << "Parse failure\n";
if (f!=l)
std::cout << "Remaining unparsed input: '" << std::string(f,l) << "'\n";
}
Prints:
Parse success: {int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }
With debug info
<startRule>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>int main() { if(true</try>
<identifier_soup>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<success>() { if(true) { std:</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>() { if(true) { std:</try>
<nestedSomething>
<try>) { if(true) { std::</try>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>) { if(true) { std::</try>
<fail/>
</parenthesized>
<block>
<try>) { if(true) { std::</try>
<fail/>
</block>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> { if(true) { std::c</success>
<attributes>[]</attributes>
</parenthesized>
<block>
<try> { if(true) { std::c</try>
<startRule>
<try> if(true) { std::cou</try>
<ifElseNormalRule>
<try> if(true) { std::cou</try>
<statement>
<try> { std::cout << "Yes</try>
<block>
<try> { std::cout << "Yes</try>
<startRule>
<try> std::cout << "Yes\n</try>
<ifElseNormalRule>
<try> std::cout << "Yes\n</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "Yes\n</try>
<identifier_soup>
<try>std::cout << "Yes\n"</try>
<ifElseNormalRule>
<try>std::cout << "Yes\n"</try>
<fail/>
</ifElseNormalRule>
<success>} else { std::cout <</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} else { std::cout <</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} else { std::cout <</try>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} else { std::cout <</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</startRule>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</block>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</statement>
<statement>
<try> { std::cout << "No\</try>
<block>
<try> { std::cout << "No\</try>
<startRule>
<try> std::cout << "No\n"</try>
<ifElseNormalRule>
<try> std::cout << "No\n"</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "No\n"</try>
<identifier_soup>
<try>std::cout << "No\n";</try>
<ifElseNormalRule>
<try>std::cout << "No\n";</try>
<fail/>
</ifElseNormalRule>
<success>} }</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} }</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} }</try>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</startRule>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</block>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</statement>
<success> }</success>
<attributes>[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]</attributes>
</ifElseNormalRule>
<ifElseNormalRule>
<try> }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> }</try>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>}</try>
<fail/>
</parenthesized>
<block>
<try>}</try>
<fail/>
</block>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> }</success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</startRule>
<success></success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success></success>
<attributes>[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try></try>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try></try>
<fail/>
</parenthesized>
<block>
<try></try>
<fail/>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success></success>
<attributes>[[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]]</attributes>
</startRule>
Summary/advice
If you ran the above sample on its own source you'll notice it bails on the namespace in line 9. Though it's nice that the program completes, rather than crashing, that's not encouraging.
You need a proper grammar. You need to THINK about what is a statement, what is a block, what is a keyword and what is the relationship between them. Part of your confusion seems to come from conflating expressions with statements.
I'd very much consider coming up with a grammar that you 1. can work 2. you know why it will work instead of ... seemingly just trying some things to cheat your way through parsing a programming language.
Have you even considered how you'd parse std::cout << "int main() { return 0; }"; yet?
Have you thought about macros with line-continuations?
