Macros and string interpolation (Julia)

Question

Let's say I make this simple string macro

macro e_str(s)
    return string("I touched this: ",s)
end

If I apply it to a string with interpolation, I obtain:

julia> e"foobar $(log(2))"
"I touched this: foobar \$(log(2))"

Whereas I would like to obtain:

julia> e"foobar $(log(2))"
"I touched this: foobar 0.6931471805599453"

What changes do I have to make to my macro declaration?

Maybe return string("I touched this: ",eval(parse("\""*s*"\""))). Care needed to get the context of evaluation right in case $ expression uses variables. — Dan Getz
Don't use eval in macros! This definitely needs to be made easier. A better version of your comment is return :(string("I touched this: ", $(parse("\""*s*"\"")))), but it'll break if s contains any " literals. — mbauman
Thanks @MattB. Definitely better. Forgot to hook up the alarm bell sounding whenever eval is used. Perhaps doing "\"\"\""*s*"\"\"\"" would reduce " risk (even more elaborately, replacing " with \" could be attempted). — Dan Getz
Why do you want to use a macro? Can't you just do this with standard string interpolation? — David P. Sanders
@DavidP.Sanders I could, but I fancied a macro, and I found this interesting enough to ask in SO. I'm learning about macros, and things like this confuse me. — RedPointyJackson

Fengyang Wang Fengyang Wang · Accepted Answer · 2016-09-14T20:47:36

It's better to parse the string at compile-time than to delegate to Julia. Basically, put the string into an IOBuffer, scan the string for $ signs, and use the parse function whenever they come up.

macro e_str(s)
    components = []
    buf = IOBuffer(s)
    while !eof(buf)
        push!(components, rstrip(readuntil(buf, '$'), '$'))
        if !eof(buf)
            push!(components, parse(buf; greedy=false))
        end
    end
    quote
        string($(map(esc, components)...))
    end
end

This doesn't work with escaped $ characters, but that can be resolved with some minor changes to handle \ also. I have included a basic example at the bottom of this post.

I wrote it this way because string macros are generally not for emulating Julia strings — regular macros with regular string literals are better for that purpose. So writing up the parsing yourself isn't that bad, especially because it allows customized extensions. If you really want parsing to be identical to how Julia parses it, you could escape the string and then reparse it, as @MattB suggested:

macro e_str(s)
    esc(parse("\"$(escape_string(s))\""))
end

The resulting expression is a :string expression which you could dump and inspect, and then analyse the usual way.

String macros do not come with built-in interpolation facilities. However, it is possible to manually implement this functionality. Note that it is not possible to embed without escaping string literals that have the same delimiter as the surrounding string macro; that is, although """ $("x") """ is possible, " $("x") " is not. Instead, this must be escaped as " $(\"x\") ".

There are two approaches to implementing interpolation manually: implement parsing manually, or get Julia to do the parsing. The first approach is more flexible, but the second approach is easier.

Manual parsing

macro interp_str(s)
    components = []
    buf = IOBuffer(s)
    while !eof(buf)
        push!(components, rstrip(readuntil(buf, '$'), '$'))
        if !eof(buf)
            push!(components, parse(buf; greedy=false))
        end
    end
    quote
        string($(map(esc, components)...))
    end
end

Julia parsing

macro e_str(s)
    esc(parse("\"$(escape_string(s))\""))
end

This method escapes the string (but note that escape_string does not escape the $ signs) and passes it back to Julia's parser to parse. Escaping the string is necessary to ensure that " and \ do not affect the string's parsing. The resulting expression is a :string expression, which can be examined and decomposed for macro purposes.

Macros and string interpolation (Julia)

1 Answers

Manual parsing

Julia parsing