Boost spirit x3 parser throws std::logic_error while parsing

Question

I created a grammar using boost spirit x3. While testing my resulting parser, i recognized that there is a case where the parser throws the following exeption:

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid

I didnt know this could happen using boost spirit x3, I thought the parse method eighter returns false or throws a boost::spirit::x3::expectation_failure, what am I doing wrong in my grammar, because this should not happen here. I uploaded my mcve HERE, if you execute the binary the std::logic error will bw thrown.

example.cpp

#include "example_def.hpp"
#include "config.hpp"

namespace client { namespace parser {

BOOST_SPIRIT_INSTANTIATE(var_dec_type, iterator_type, context_type)

}}

main.cpp

#include <iostream>
//#define SINGLE_TU
#if defined(SINGLE_TU)
#pragma message "yesdef(SINGLE_TU)"
#else
#pragma message "notdef(SINGLE_TU)"
#endif
#ifdef SINGLE_TU
#include "types_def.hpp"
#include "example_def.hpp"
#else
#include "example.hpp"
#endif

template<typename Parser, typename Attribute>
bool test(const std::string& str, Parser&& p, Attribute&& attr)
{
    using iterator_type = std::string::const_iterator;
    iterator_type in = str.begin();
    iterator_type end = str.end();

    bool ret = boost::spirit::x3::phrase_parse(in, end, p, boost::spirit::x3::ascii::space, attr);
    ret &= (in == end);
    return ret;

}

int main()
{
    client::ast::VariableDec attr;

    std::cout << test("var foo : foo<bar, baz<bahama>>", client::var_dec() , attr);
    return 0;
}

types.cpp

#include "config.hpp"
#include "types_def.hpp"
#include <iostream>

namespace client {namespace parser {

BOOST_SPIRIT_INSTANTIATE(type_type, iterator_type, context_type);

BOOST_SPIRIT_INSTANTIATE(class_type_type, iterator_type, context_type);
#define PARSE_RULE_BODY
#if defined(PARSE_RULE_BODY)
  #pragma message "yesdef(PARSE_RULE_BODY)"
  //This causes the parse_rule generated by
  //BOOST_SPIRIT_DEFINE for type
  //to *not* be used.
  #include <boost/fusion/iterator/deref.hpp>
#else
  #pragma message "notdef(PARSE_RULE_BODY)"
  //This causes  the parse_rule generated by
  //BOOST_SPIRIT_DEFINE for type
  //to be used.
#endif
#define SEHE_TYPES_DEF_HPP_PASTE
#define EXPLICT_SPECIALIZATION_FROM_LINKER_ERROR_MSG
#if defined(SEHE_TYPES_DEF_HPP_PASTE)
#pragma message "yesdef(SEHE_TYPES_DEF_HPP_PASTE)"
//Source:
//  The following code was copy&pasted&reformatted from lines 38-53 of:
//   https://github.com/sehe/linker_error_example/blob/explicit_instantiation/types_def.hpp
//  on about 2016-11-15.
//  with minor modifications:
//    * reformatting
//    * Added body with:
//      * printing first..last.
//=======================
namespace {
      template <typename Seq, size_t N>
    using field_at = 
      boost::fusion::basic_iterator
      < boost::fusion::struct_iterator_tag
      , boost::fusion::random_access_traversal_tag
      , Seq
      , N
      >;
      template <typename Seq, size_t N, size_t M>
    using fields = 
      boost::fusion::iterator_range<field_at<Seq, N>, field_at<Seq, M> >;
    using Attributes = 
      fields<client::ast::VariableDec, 1, 2>;
    using Context = 
      x3::context
      < x3::skipper_tag
      , x3::char_class<boost::spirit::char_encoding::ascii, x3::space_tag> const
      , x3::unused_type
      >;
}
template
  #ifdef PARSE_RULE_BODY
    <> 
  #endif
    bool parse_rule
    < std::string::const_iterator
    , Context
    , Attributes
    >(
    decltype(type) rule,
    std::string::const_iterator &first, 
    std::string::const_iterator const &last,
    Context const &context,
    Attributes &attr)
    #ifndef PARSE_RULE_BODY
      ;
    #else
      {
        std::cout<<"***in function="<<__func__<<"\n";
        std::string::const_iterator beg=first;
        std::cout<<":input=\n{";
        for(; beg!=last; ++beg) std::cout<<*beg;
        std::cout<<"}\n";
        using required_t=type_type::attribute_type;
        required_t required_v=boost::fusion::deref(attr.first);
          //attr type required by parse_rule generated by:
          //  BOOST_SPIRIT_INSTANTIATE(type_type, iterator_type, context_type)
          ;
        bool result=parse_rule(rule,first,last,context,required_v)
          //This should call the parse_rule generated by:
          //  BOOST_SPIRIT_INSTANTIATE(type_type, iterator_type, context_type)
          ;
        std::cout<<":result="<<result<<"\n";
        return result;
      }
    #endif

#elif defined(EXPLICT_SPECIALIZATION_FROM_LINKER_ERROR_MSG)
#pragma message "yesdef(EXPLICT_SPECIALIZATION_FROM_LINKER_ERROR_MSG)"
template
  #ifdef PARSE_RULE_BODY
    <> 
  #endif
  //The following simply copied&pasted from an earlier linker error
  //message, and then reformatted to clarify what
  //was being specialized.
  bool
  parse_rule
  < __gnu_cxx::__normal_iterator
    < char const*
    , std::__cxx11::basic_string
      < char
      , std::char_traits<char>
      , std::allocator<char> 
      > 
    >
  , boost::spirit::x3::context
    < boost::spirit::x3::skipper_tag
    , boost::spirit::x3::char_class
      < boost::spirit::char_encoding::ascii
      , boost::spirit::x3::space_tag
      > const
    , boost::spirit::x3::unused_type
    >
  , boost::fusion::iterator_range
    < boost::fusion::basic_iterator
      < boost::fusion::struct_iterator_tag
      , boost::fusion::random_access_traversal_tag
      , client::ast::VariableDec
      , 1
      >
    , boost::fusion::basic_iterator
      < boost::fusion::struct_iterator_tag
      , boost::fusion::random_access_traversal_tag
      , client::ast::VariableDec
      , 2
      > 
    >
  >
  ( boost::spirit::x3::rule//==decltype(type) where types from types_def.hpp:16
    < client::parser::type_class//types.hpp:15
    , boost::spirit::x3::variant
      < client::ast::nil
      , boost::spirit::x3::forward_ast
        < client::ast::LambdaType
        >
      , boost::spirit::x3::forward_ast
        < client::ast::ClassType
        > 
      >
    , false
    >
  , __gnu_cxx::__normal_iterator//==std::string::const_iterator
    < char const*
    , std::__cxx11::basic_string
      < char
      , std::char_traits<char>
      , std::allocator<char> 
      > 
    >& first
  , __gnu_cxx::__normal_iterator//==std::string::const_iterator
    < char const*
    , std::__cxx11::basic_string
      < char
      , std::char_traits<char>
      , std::allocator<char> 
      > 
    > const& last
  , boost::spirit::x3::context//=?Context from #if defined(SEHE_TYPES_DEF_HPP_PASTE)
    < boost::spirit::x3::skipper_tag
    , boost::spirit::x3::char_class
      < boost::spirit::char_encoding::ascii
      , boost::spirit::x3::space_tag
      > const
    , boost::spirit::x3::unused_type
    > const& context
  , boost::fusion::iterator_range//=?Attributes from #if defined(SEHE_TYPES_DEF_HPP_PASTE)
    < boost::fusion::basic_iterator
      < boost::fusion::struct_iterator_tag
      , boost::fusion::random_access_traversal_tag
      , client::ast::VariableDec
      , 1
      >
    , boost::fusion::basic_iterator
      < boost::fusion::struct_iterator_tag
      , boost::fusion::random_access_traversal_tag
      , client::ast::VariableDec
      , 2
      > 
    >& attr
  )
#ifndef PARSE_RULE_BODY
  ;
#else
  {
    std::cout<<"***in function="<<__func__<<"\n";
    std::string::const_iterator beg=first;
    std::cout<<":input=\n{";
    for(; beg!=last; ++beg) std::cout<<*beg;
    std::cout<<"}\n";
    return false;
  }
#endif//PARSE_RULE_BODY
#endif//(SEHE_TYPES_DEF_HPP_PASTE)
}}

ast.hpp

//
// Created by lukas on 11.11.16.
//

#ifndef LINKER_ERROR_EXAMPLE_AST_HPP_HPP
#define LINKER_ERROR_EXAMPLE_AST_HPP_HPP

#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>

#include <vector>
#include <string>

namespace client { namespace ast {

namespace x3 = boost::spirit::x3;

struct LambdaType;
struct ClassType;
struct nil{};

typedef x3::variant <
    nil,
    x3::forward_ast<LambdaType>,
    x3::forward_ast<ClassType>
> Type;

struct LambdaType {
    std::vector<Type> parameters_;
    Type return_type_;
};

struct ClassType {
    std::vector<std::string> name_;
    std::vector<Type> template_args_;
};

struct VariableDec {
    std::string _name;
    Type _type;
};


}}

BOOST_FUSION_ADAPT_STRUCT(client::ast::LambdaType, parameters_, return_type_)
BOOST_FUSION_ADAPT_STRUCT(client::ast::ClassType, name_, template_args_)
BOOST_FUSION_ADAPT_STRUCT(client::ast::VariableDec, _name, _type)

#endif //LINKER_ERROR_EXAMPLE_AST_HPP_HPP

config.hpp

#ifndef LINKER_ERROR_EXAMPLE_CONFIG_HPP
#define LINKER_ERROR_EXAMPLE_CONFIG_HPP

#include <boost/spirit/home/x3.hpp>

namespace client{
namespace parser{

namespace x3 = boost::spirit::x3;


typedef std::string::const_iterator iterator_type;
typedef x3::phrase_parse_context<x3::ascii::space_type>::type context_type;

}
}

#endif //LINKER_ERROR_EXAMPLE_CONFIG_HPP

example_def.hpp

#ifndef LINKER_ERROR_EXAMPLE_EXAMPLE_DEF_HPP
#define LINKER_ERROR_EXAMPLE_EXAMPLE_DEF_HPP

#include "ast.hpp"
#include "example.hpp"
#include "types.hpp"

namespace client { namespace parser {

#ifndef SINGLE_TU
namespace { const auto& type = client::type(); }
#endif

const var_dec_type var_dec = "var_dec";

#define EXAMPLE_DEF_LINK_ERR
#if defined(EXAMPLE_DEF_LINK_ERR)
#pragma message "yesdef(EXAMPLE_DEF_LINK_ERR)"
#else
#pragma message "notdef(EXAMPLE_DEF_LINK_ERR)"
#endif
auto const var_dec_def = x3::lexeme["var "]
                         > +x3::alnum
                         > ":"
                       #ifdef EXAMPLE_DEF_LINK_ERR 
                         >> type //<- this gets linker error.
                       #else
                         > type //<- This links.
                       #endif
                         > ";";

BOOST_SPIRIT_DEFINE(
  var_dec
)

}}

namespace client {

const parser::var_dec_type& var_dec()
{
    return parser::var_dec;
}

}

#endif //LINKER_ERROR_EXAMPLE_EXAMPLE_DEF_HPP

example.hpp

#ifndef LINKER_ERROR_EXAMPLE_EXAMPLE_HPP
#define LINKER_ERROR_EXAMPLE_EXAMPLE_HPP

#include <boost/spirit/home/x3.hpp>

#include "ast.hpp"

namespace client { namespace parser {

namespace x3 = boost::spirit::x3;

class var_dec_class {};

typedef x3::rule<var_dec_class, ast::VariableDec> var_dec_type;

BOOST_SPIRIT_DECLARE(var_dec_type)


}}

namespace client {

const parser::var_dec_type& var_dec();

}
#endif //LINKER_ERROR_EXAMPLE_EXAMPLE_HPP

types_def.hpp

#ifndef KYLE_TYPES_DEF_HPP
#define KYLE_TYPES_DEF_HPP

#include "types.hpp"


namespace client { namespace parser {
namespace x3 = boost::spirit::x3;


typedef x3::rule<struct lambda_type_class, ast::LambdaType> lambda_type_type;


const class_type_type class_type = "class_type";
const lambda_type_type lambda_type = "lambda_type";
const type_type type = "type";

auto const identifier = +x3::alnum;

auto const type_def =
        (lambda_type | class_type);

auto const lambda_type_def =
        ("(" > -(type % ",") > ")" > "=>" > type)
        | (x3::repeat(1)[class_type] >> "=>" > type);


auto const class_type_def =
        (identifier % "::") >> -("<" > type % "," > ">");


BOOST_SPIRIT_DEFINE(
  lambda_type,
  class_type,
  type
)


}}

namespace client {

const parser::class_type_type& class_type()
{
    return parser::class_type;
}
const parser::type_type& type()
{
    return parser::type;
}

}

#endif //KYLE_TYPES_DEF_HPP

types.hpp

#ifndef KYLE_PARSER_TYPES_HPP
#define KYLE_PARSER_TYPES_HPP

#include <boost/spirit/home/x3.hpp>

#include "ast.hpp"

namespace client { namespace parser {
namespace x3 = boost::spirit::x3;

struct class_type_class;
struct type_class;

typedef x3::rule<class_type_class, ast::ClassType> class_type_type;
typedef x3::rule<type_class, ast::Type> type_type;

BOOST_SPIRIT_DECLARE(class_type_type,
                     type_type)


}}


namespace client {

const parser::class_type_type& class_type();
const parser::type_type& type();

}

#endif

An example with ten files might be complete and verifiable, but it's certainly not minimal. — eh9
Please remove all those kludges related to your old linker error. They're irrelevant here. I'm not at all flattered by being referenced in top of some 200 lines of horrible template diarrhea that I most definitely never wrote. Here's what I showed you instead. — sehe

sehe sehe · Accepted Answer · 2016-11-18T10:37:44

It throws an expectation_failed exception.

Quite obviously it's because you have a missing ;.

After cleaning out the horrendous kludges that were related to the old linker-error workarounds, I simply changed the example_def expression to:

auto const var_dec_def = x3::lexeme["var "] > +x3::alnum >> (":" > type) > ";";

Now, add proper error handling:

int main()
{
    client::ast::VariableDec attr;
    std::string const input("var foo : foo<bar, baz<bahama>>;");

    try {
        std::cout << test(input, client::var_dec() , attr);
    } catch(boost::spirit::x3::expectation_failure<std::string::const_iterator> const& e) {
        std::cout << "Expected: " << e.which() << " at '" << std::string(e.where(), input.end()) << "'\n";
        return 255;
    }
}

Prints:

Leaving out the trailing ; prints:

Expected: ';' at ''

Full demo: error-handling branch

 CMakeLists.txt  |   6 +-
 example.cpp     |   9 +++
 example_def.hpp |  33 +--------
 main.cpp        |  20 ++----
 types.cpp       | 203 ++++----------------------------------------------------
 types_def.hpp   |  20 ++----
 6 files changed, 38 insertions(+), 253 deletions(-)