Racket: Macros generated by macros defined as macroname.0

Question

I'm trying to learn a bit about Racket and its macro system. My introduction has been writing a thin wrapper around the IEX stock API. My prep reading included Greg Hendershott's Fear of Macros, the Racket Guide (in particular, the section on macro-generating macros and surrounding documentation), and other online resources.

I'd like to be able to do something like this:

(iex-op* chart dividends quote)

to generate the functions iex-chart, iex-dividends, et c. The use of a syntax transformer seemed like a natural fit to me since I don't think I could easily write a function-generating function with a clean syntax and multiple arguments.

So:

(define-syntax-rule (iex-op* op0 ...)
  (begin
    (iex-op op0) ...))

This assumes a working iex-op syntax transformer:

(define iex-base-url (string->url "https://api.iextrading.com"))
(define iex-ver "1.0")

(define-syntax (iex-op stx)
  (syntax-case stx (quote) ; quote is one of the operations
    [(_ oper)
     (with-syntax ([op-name (format-id stx "iex-~a" #'oper)])
         #'(define (op-name ticker . args)
             (let ([op-url
                    (combine-url/relative
                        iex-base-url
                        (string-join
                          `(,iex-ver
                            "stock"
                            ,(symbol->string ticker)
                            ,(symbol->string (syntax-e #'oper))
                            ,(string-join args "/")) "/"))])
              (string->jsexpr (http-response-body (get http-requester op-url))))))]))

My problem arises not with the iex-op macro, which appears to do the Right Thing, but with the use of iex-op*, which does not:

Welcome to Racket 6.3
> (enter! "iex.rkt")
> (iex-op quote)
> iex-<TAB><TAB>
iex-base-url iex-op       iex-op*      iex-quote    iex-ver
> (iex-op* chart dividends)
> iex-<TAB><TAB>
iex-base-url    iex-dividends.0 iex-op*         iex-ver
iex-chart.0     iex-op          iex-quote

The op*-defined functions are suffixed with .0. I don't know why, and I can't find documentation about it conveniently despite some hours of searching.

When I run the macro expander in DrRacket, I find that (iex-op* chart dividends) actually does expand to

(begin (iex-op chart) (iex-op dividends))

as desired. What's worse, when I reproduce the results of the syntax transformation in the REPL, it works!

> (begin (iex-op chart) (iex-op dividends))
> iex-<TAB><TAB>
iex-base-url    iex-chart.0     iex-dividends.0 iex-op*         iex-ver
iex-chart       iex-dividends   iex-op          iex-quote

What am I missing? I'll readily admit my code probably needs some substantial clean-up (I'm slowly bending my Python/C/etc. mind), but I'm rather less concerned about its aesthetic and more about what arcanum is causing this behavior.

Alexis King Alexis King · Accepted Answer · 2018-05-28T05:54:19

The .0 suffix isn’t actually a suffix on the name of the generated binding—it indicates that the identifier was introduced by a macro and lives inside a macro scope. The root problem is your use of format-id.

Remember that Racket’s macro system is hygienic. This means that bindings introduced by macros can only be seen by macros. This makes autocomplete in the REPL a little more complicated, and I don’t think what the REPL does here is honestly a very helpful choice. That said, what you probably want to know is how to fix your code, not the intricacies of why the REPL shows macro-introduced bindings with a .0 suffix.

When you call format-id, the produced identifier will have the same lexical context as the first argument. You can think of this as essentially meaning that the produced identifier will live in the same lexical scope as that piece of syntax. In your case, you are providing stx, which represents the entire input form to the macro.

When you use iex-op directly, by writing (iex-op quote), for example, then the quote identifier comes from the same lexical scope as the containing (iex-op ....) form. Therefore, when you call format-id and give it the (iex-op ....) form as the first argument, you still get an identifier that shares the same lexical scope as the quote identifier, and all is well.

However, when you use your iex-op* macro, it passes the op0 syntax object through, but the surrounding (iex-op ....) form comes from the iex-op* macro. This means that stx now refers to the scope of the inside of the iex-op* macro, not the scope of the op0 identifier. To fix this, change your call to format-id so that it creates an identifier with the same scope as the operation identifier rather than the surrounding syntax object:

(format-id #'oper "iex-~a" #'oper)

Now your macro will work as intended.

Before finishing up, ask yourself: what are some takeaways here? Here are a couple:

Breaking hygiene is subtle. When you decide to write an unhygienic macro (and format-id is unhygienic), think very hard about where you’re getting the lexical context from. Consider what will happen when users write macros over your macro.

In this case, you’re lucky that you are the author of both iex-op and iex-op*, so you could fix iex-op after you ran into a problem with it. But if you were not the author of iex-op, only iex-op*, you would be in a much trickier situation. Think about these things whenever you write any unhygienic macro to avoid such problems down the line.
As a rule of thumb, when unhygienically forging a new identifier from another identifier, it’s best to use that other identifier as the source of the lexical context. That way, the lexical scope of the identifier the user provided will be preserved, which is what users expect.

Arguably, some of the examples in Fear of Macros that use format-id are sloppy in this respect. It might be worth trying to improve them to set a better example.

Racket: Macros generated by macros defined as macroname.0

1 Answers