Using the auto_ expression in boost::spirit with std::vectors

Question

I'm pretty new to boost::spirit. I would like to parse a string of comma separated objects into an std::vector (similarly as in the tutorials). The string could be of different types (known at compile time): integers, like "1,2,3", strings "Apple, Orange, Banana", etc. etc. I would like to have a unified interface for all types.

If I parse a single element I can use the auto_ expression. Is it possible to have a similar interface with vectors? Can I define a rule that, given a template parameter, can actually parse this vector?

Here is a simple sample code (which does not compile due to the last call to phrase_parse):

#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>

#include <iostream>
#include <vector>
#include <boost/spirit/include/qi_auto.hpp>


namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
namespace phoenix = boost::phoenix;

using qi::auto_;
using qi::phrase_parse;
using ascii::space;
using phoenix::push_back;


int main()
{
    std::string line1 = "3";
    std::string line2 = "1, 2, 3";

    int v;
    std::vector<int> vector;

    typedef std::string::iterator stringIterator;

    stringIterator first = line1.begin();
    stringIterator last  = line1.end();

    namespace qi    = boost::spirit::qi;
    namespace ascii = boost::spirit::ascii;

    bool r1 = qi::phrase_parse( first,
                                last,
                                qi::auto_,
                                ascii::space,
                                v );

    first = line2.begin();
    last  = line2.end();

    //The following call is wrong!
    bool r2 = qi::phrase_parse( first,
                                last,
                              //  Begin grammar
                               (
                                qi::auto_[push_back(phoenix::ref(vector), qi::_1)]
                                >> *(',' >> qi::auto_[push_back(phoenix::ref(vector),qi::_1)])
                               ),
                              //  End grammar
                                ascii::space, 
                                vector);
    return 0;
}

UPDATE

I found a solution, in the case the size of the vector is known before parsing. On the other hand I cannot use the syntax *( ',' >> qi::auto_ ).

#include <boost/spirit/include/qi.hpp>

namespace qi     = boost::spirit::qi;

int main()
{
    std::string s = "1, 2, 3";

    std::vector<int> vector;
    //This works
    qi::phrase_parse(s.begin(), s.end(), qi::auto_ >> ',' >> qi::auto_  >> ',' >> qi::auto_  , qi::blank, vector);
    //This does not compile
    qi::phrase_parse(s.begin(), s.end(), qi::auto_ >> *( ',' >> qi::auto_ ) , qi::blank, vector);

    for(int i = 0; i < vector.size() ; i++)
        std::cout << i << ": " << vector[i] << std::endl;

    return 0;
}

Moreover using auto_, I cannot parse a a string. Is it possible to define e template function, where the grammar can be deduced by the template parameter?

template< typename T >
void MyParse(std::string& line, std::vector<T> vec)
{
     qi::phrase_parse( line.begin(), 
                       line.end(), 
                       /* 
                       How do I define a grammar based on T
                       such as: 
                          double_ >> *( ',' >> double_ )    for T = double
                       +qi::alnum >> *( ',' >> +qi::alnum ) for T = std::string
                       */, 
                       qi::blank, 
                       vec);
}

Ignoring the problem with std::string, what you want to use is the list parser ( auto_%',' ). — llonesmiz
Looking at the example here, this seems to work (you could change +(char_-',') to +alnum if you prefer). — llonesmiz
@cv_and_he It works like a charm! The spirit documentation of the list parser claims that for ` a % b ` : "This is equivalent to a >> *(b >> a)". But with ` auto_ ` it is different, as it does not even compile. — srossi
auto_ isn't a good match for std::string, because it would match anything, really — sehe
@cv_and_he I've come up with a slightly less intrusive way to deal with std::string vs. auto_ I think in my updated answer. You should probably post that create_parser<> based version as an answer too... It's worth the upvotes. — sehe

sehe sehe · Accepted Answer · 2015-01-25T16:41:51

auto_ has support for container attributes out of the box:

Live On Coliru

std::istringstream iss("1 2 3 4 5; 6 7 8 9;");
iss.unsetf(std::ios::skipws);

std::vector<int> i;
std::vector<double> d;

if (iss >> qi::phrase_match(qi::auto_ >> ";" >> qi::auto_, qi::space, i, d))
{
    for (auto e:i) std::cout << "int: " << e << "\n";
    for (auto e:d) std::cout << "double: " << e << "\n";
}

Prints

int: 1
int: 2
int: 3
int: 4
int: 5
double: 6
double: 7
double: 8
double: 9

So you could basically write your template function by using ',' as the skipper. I'd prefer the operator% variant though.

Simple Take

template<typename Container>
void MyParse(std::string const& line, Container& container)
{
    auto f(line.begin()), l(line.end());
    bool ok = qi::phrase_parse(
            f, l,
            qi::auto_ % ',', qi::blank, container);

    if (!ok || (f!=l))
        throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}

Variant 2

template<typename Container>
void MyParse(std::string const& line, Container& container)
{
    auto f(line.begin()), l(line.end());
    bool ok = qi::phrase_parse(
            f, l,
            qi::auto_, qi::blank | ',', container);

    if (!ok || (f!=l))
        throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}

Solving the `string` case (and others):

If the element type is not 'deducible' by Spirit (anything could be parsed into a string), just take an optional parser/grammar that knows how to parse the element type?

template<typename Container, typename ElementParser = qi::auto_type>
void MyParse(std::string const& line, Container& container, ElementParser const& elementParser = ElementParser())
{
    auto f(line.begin()), l(line.end());
    bool ok = qi::phrase_parse(
            f, l,
            elementParser % ",", qi::blank, container);

    if (!ok || (f!=l))
        throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}

Now, it parses strings just fine:

std::vector<int> i;
std::set<std::string> s;

MyParse("1,22,33,44,15", i);
MyParse("1,22,33,44,15", s, *~qi::char_(","));

for(auto e:i) std::cout << "i: " << e << "\n";
for(auto e:s) std::cout << "s: " << e << "\n";

Prints

i: 1
i: 22
i: 33
i: 44
i: 15
s: 1
s: 15
s: 22
s: 33
s: 44

Full Listing