How to annotate the AST with position information?

Question

I'm using Boost Spirit to parse source files in a little compiler project.

If an error arise during the parsing itself, I can print the position of the error, but how do I do in later phase, typically when performing semantic checks ?

My source file is parsed into an Abstract Syntax Tree using auto rules. I want to add line and col informations into the AST nodes. Is there a easy way to achieve that during parsing ?

I'm using boost::spirit::classic::position_iterator2 in my Lexer and then use this lexer in my grammar.

Thank you

EDIT for sehe:

The lexer is defined like that:

typedef std::string::iterator base_iterator_type;
typedef boost::spirit::classic::position_iterator2<base_iterator_type> pos_iterator_type;
typedef boost::spirit::lex::lexertl::token<pos_iterator_type> Tok;
typedef boost::spirit::lex::lexertl::actor_lexer<Tok> lexer_type;

template<typename L>
class SpiritLexer : public lex::lexer<L> {
   //Token definitions
}

typedef lexer_type::iterator_type Iterator;
typedef SpiritLexer<lexer_type> Lexer;

The grammars are defined like that:

struct EddiGrammar : qi::grammar<lexer::Iterator, ast::SourceFile()> {
    EddiGrammar(const lexer::Lexer& lexer);

    //Token definitions
};

And finally, here is how I parse the source:

ast::SourceFile program

std::ifstream in(file.c_str());
in.unsetf(std::ios::skipws);

in.seekg(0, std::istream::end);
std::size_t size(static_cast<size_t>(in.tellg()));

in.seekg(0, std::istream::beg);

std::string contents(size, 0);
in.read(&contents[0], size);

pos_iterator_type position_begin(contents.begin(), contents.end(), file);
pos_iterator_type position_end;

Lexer lexer;
EddiGrammar grammar(lexer);

bool r = spirit::lex::tokenize_and_parse(position_begin, position_end, lexer, grammar, program);

In my grammar, I use the lexer by referring to some lexer tokens. For example:

else_ %=
        lexer.else_
    >>  lexer.left_brace
    >>  *(instruction)
    >>  lexer.right_brace;

All my AST nodes are constructed using auto rules.

i have spend some time a few days back. A small sample of how exactly you are using Spirit Lex would be helpful. — sehe
I added samples on how I use Spirit Lex from the Spirit Parser. I hope this is enough ? — Baptiste Wicht

Baptiste Wicht Baptiste Wicht · Accepted Answer · 2012-02-26T18:23:07

I've found a way to fix this problem.

I wrote a simple terminal parser that just get the current Position and add it to AST node.

Here is the parser I wrote:

namespace boost { namespace spirit {
    BOOST_SPIRIT_TERMINAL(position)

    template <>
    struct use_terminal<qi::domain, tag::position> : mpl::true_ {};
}}

namespace boost { namespace spirit { namespace qi
{
    struct position : primitive_parser<position>
    {
        position(const eddic::lexer::pos_iterator_type& position_begin) : position_begin(position_begin) {}

        template <typename Context, typename Iterator>
        struct attribute {
            typedef eddic::ast::Position type;
        };

        template <typename Iterator, typename Context
            , typename Skipper, typename Attribute>
            bool parse(Iterator& first, Iterator const& last
                    , Context& /*context*/, Skipper const& skipper, Attribute& attr) const
        {
            qi::skip_over(first, last, skipper);

            auto& pos = position_begin.get_position();

            attr.theLine = position_begin.get_currentline();
            attr.file = pos.file;
            attr.column = pos.column;
            attr.line = pos.line;

            return true;
        }

        template <typename Context>
        info what(Context& context) const {
            return info("position");
        }

        const eddic::lexer::pos_iterator_type& position_begin;
    };

    template <typename Modifiers>
    struct make_primitive<tag::position, Modifiers> {
        typedef position result_type;
        result_type operator()(unused_type, eddic::lexer::Lexer const& lexer, unused_type) const
        {
            return result_type(lexer);
        }
    };
}}}

and the struct I use to store the information:

struct Position {
    std::string file;
    std::string theLine;
    int line;
    int column;
};

It works well, but I have to pass the position iterator to the parser. If someone know a way to get the position_iterator2 iterator from the Iterator provided to the parse function, I will appreciate.

How to annotate the AST with position information?

1 Answers