How to test a clojure macro that uses gensyms?

Question

I want to test a macro that uses gensyms. For example, if I want to test this:

(defmacro m1
  [x f]
  `(let [x# ~x]
    (~f x#)))

I can use macro-expansion...

(macroexpand-1 '(m1 2 inc))

...to get...

(clojure.core/let [x__3289__auto__ 2] (inc x__3289__auto__))

That's easy for a person to verify as being correct.

But how can I test this in an practical, clean automated fashion? The gensym is not stable.

(Yes, I know the particular macro example is not compelling, but the question is still fair.)

I realize Clojure expressions can be treated as data (it is a homoiconic language), so I can pull apart the result like this:

(let [result (macroexpand-1 '(m1 2 inc))]
  (nth result 0) ; clojure.core/let
  (nth result 1) ; [x__3289__auto__ 2]
  ((nth result 1) 0) ; x__3289__auto__
  ((nth result 1) 0) ; 2
  (nth result 2) ; (inc x__3289__auto__)
  (nth (nth result 2) 0) ; inc
  (nth (nth result 2) 1) ; x__3289__auto__
  )

But this is unwieldy. Are there better ways? Perhaps there are data structure 'validation' libraries that could come in handy? Maybe destructuring would make this easier? Logic programming?

UPDATE / COMMENTARY:

While I appreciate the advice of experienced people who say "don't test the macro-expansion itself", it doesn't answer my question directly.

What is so bad about "unit testing" a macro by testing the macro-expansion? Testing the expansion is reasonable -- and in fact, many people test their macros that way "by hand" in the REPL -- so why not test it automatically too? I don't see a good reason to not do it. I admit that testing the macro-expansion is more brittle than testing the result, but doing the former can still have value. You can also test the functionality as well -- you can do both! This isn't an either/or decision.

Here is my psychological explanation. One of the reasons that people don't test the macro-expansion is that it currently is a bit of a pain. In general, people often rationalize against doing something when it seems difficult, independent of its intrinsic value. Yes -- that is exactly why I asked this question! If it were easy, I think people would do it more often. If it were easy, they would be less likely to rationalize by giving answers saying that "it is not worth doing."

I also understand the argument that "you should not write a complex macro". Sure. But let's hope people do not go as far as to think "if we encourage a culture of not testing macros, then that will prevent people from writing complex ones." Such an argument would be silly. If you have a complex macro-expansion, testing that it works as you expect is a sane thing to do. I am personally not beneath testing even simple things, because I am often surprised that bugs can come from simple mistakes.

Answer 1

Don't test how it works (its expansion), test that it works. If you test the particular expansion, you are chained to that implementation strategy; instead, just test that (m1 2 inc) returns 3, and whatever other test cases are necessary to comfort your conscience, and then you can be happy that your macro is working.

Answer 2

This can be done with metadata. Your macro outputs a list, which can have metadata attached to it. Simply add the gensym->var mappings to that and then use them for testing.

So your macro would look something like this:

(defmacro m1 [x f]
  (let [xsym (gensym)]
    (with-meta 
      `(let [~xsym ~x]
         (~f ~xsym))
      {:xsym xsym})))

The output from the macro now has a map against it with the gensyms:

(meta (macroexpand-1 '(m1 a b)))
=> {:xsym G__24913}

To test the macro expansion you would do something like this:

(let [out (macroexpand-1 `(m1 a b))
      xsym (:xsym (meta out))
      target `(clojure.core/let [~xsym a] (b ~xsym))]

  (= out target))

To address the question why you would want to do this: The way I normally write a macro is to generate the target code first (i.e. what I want the macro to output), test that does the right thing, then generate the macro from that. Having known-good code up front allows me to do TDD against the macro; in particular I can tweak the macro, run the tests, and if they fail clojure.test will show me the actual vs. target and I can visually inspect.