clojure macro not working properly with doseq

Question

Update: Thank you everyone for your responses, but I seem to have made a bad choice using an embedded "def" in my example, which is throwing people off. This has nothing to do with def. The problem still occurs if I do not use a def. As to why I'm doing it this way -- honestly, I'm just trying to learn about macros, and this is just one of the ways that occurred to me. I'm just trying to understand how macros work. I may very well ultimately end up using a different mechanism. I also know that having multiple defs (including defmacros) for the same thing is considered bad practice, but it still seems to me this way should work.

I am re-factoring my examples:

When I write variations of a macro-generating macro in-line (with a simplified version of what I'm actually doing):

(do 
    (defmacro abc []
      `(defmacro xyz []
         ;;(def x 7)))
         (+ 7 1)))
    (abc)
    ;;(xyz)
    ;;(spit "log.txt" (format "pass 1: x=%s\n" x ) :append false))
    (spit "log.txt" (format "pass 1: results=%s\n" (xyz) ) :append false))

(do 
    (defmacro abc []
      `(defmacro xyz []
         ;;(def x 8)))
         (+ 8 1)))
    (abc)
    ;;(xyz)
    ;;(spit "log.txt" (format "pass 2: x=%s\n" x ) :append true))
    (spit "log.txt" (format "pass 1: results=%s\n" (xyz) ) :append false))

(do 
    (defmacro abc []
      `(defmacro xyz []
         ;;(def x 9)))
         (+ 9 1)))
    (abc)
    ;;(xyz)
    ;;(spit "log.txt" (format "pass 3: x=%s\n" x ) :append true))
    (spit "log.txt" (format "pass 1: results=%s\n" (xyz) ) :append false))

It gives me what I expect:

pre-refactor:
cat log.txt 
pass 1: x=7
pass 2: x=8
pass 3: x=9

post-refactor:
cat log.txt 
pass 1: results=8
pass 2: result=9
pass 3: result=10

But when I try to iterate using doseq, it only seems to give me one value:

(def int-lookup [7 8 9])

  (doseq [i (range 3)]
    (defmacro abc []
      `(defmacro xyz []
         ;;(def x ~(int-lookup i))))
         (+ 1 ~(int-lookup i))))
      (abc)
      ;;(xyz)
      ;;(spit "log.txt" (format "pass %s: x=%s\n" i x) :append (if (= i 0) false true)))
      (spit "log.txt" (format "pass %s: result=%s\n" i (xyz)) :append (if (= i 0) false true))

Output:

pre-refactor:
cat log.txt 
pass 0: x=9
pass 1: x=9
pass 2: x=9

post-refactor
cat log.txt 
pass 0: result=10
pass 1: result=10
pass 2: result=10

I've seen it give me all 7's, and all 8's too, but never mixed.

I've tried resetting the macro symbols in-between like so:

(ns-unmap *ns* 'xyz)
(ns-unmap *ns* 'x)

However, this make things even worse, sporadically generating:

CompilerException java.lang.RuntimeException: Unable to resolve symbol: xyz in this context, compiling:(/tmp/form-init2424586203535482807.clj:5:5)

I'm sort of assuming the compiler is somehow optimizing the macro def or call, so it's only actually driving it once when using doseq. If this is the case, then how would you iterate over defmacro definitions and not have this happen? I intend to have about 15 iteration is my final solution, so I really don't want to have to in-line all definitions.

Why would you want to re-define macros or vars in the first place and is there a reason for the nested macro at all? This looks pretty convoluted to me. There is also no need for the abc macro at all, you could just as well have the (defmacro xyz in there directly, unless your real world code has a need for it. — schaueho
To elaborate a little bit more: defmacro works a lot like defn, which in turn uses def. def creates a var, interns it and binds its root value, cf. http://clojure.org/vars. You seem to use def within your xyz macro mainly to rebind x. You could use set! for this, if you don't need to set the root value across all threads. I don't think there's a guarantee of what happens with all these rebindings. — schaueho
The problem isn't that much with nested macros than with the test having a defmacro inside it. defs should only occur at top-level (or in a top-level do). — cgrand
How else can I define an embedded/nested macro without using defmacro? Paul Graham has an entire chapter in "On Lisp" devoted to macro writing macros. To quote from p 213: "When several macros have deﬁnitions of a similar form, we may be able to write a macro-deﬁning macro to produce them". He has 'defmacros within 'defmacros in his examples, so I assume you should be able to do the same in clojure — vt5491

vt5491 vt5491 · Accepted Answer · 2015-03-28T00:00:16

I'm pretty sure I know what's happening. It's a classic macro thing -- distinguishing between compile-time and run-time.

I think that when the do-seq is being compiled, the compiler needs to get something to put into the '(xyz) expression:

(spit "log.txt" (format "pass %s: result=%s\n" i (xyz)) <-- Compiler: what do I put here?....

I'm assuming it will come from the run-time value of 'xyz that is set by the prior defmacro:

 (defmacro abc []
      `(defmacro xyz []  <-- I'm assuming this version will be used
         (+ 9 1)))
    (abc)

while 'abc is known at the compile-time of the do-seq, the underlying macro 'xyz in the 'abc defmacro is only known at run-time. The compiler only knows about the 'xyz symbol set from my previous run, which returns 10. Thus the compiler statically inserts this expression, and ignores the runtime version, which why I see the same value each time.

clojure macro not working properly with doseq

1 Answers