Clojure evaluation process, the reader's job and directly passed data-structures to clojures compiler/evaluator

Question

I'm totally confused about clojure's reader. The input that the reader's read function gets passed can be source file or some ASCII text directly typed into the REPL. The reader then produces a data structure and passes it to the compiler so it could be evaluated.

What I don't understand is how that data structure 'looks like' for a compiler?

read-string function does the same as the reader's read function when it gets passed a string. read-string "(+ 1 2 3)") returns (+ 1 2 3). Does it mean then that (+ 1 2 3) is the exact representation which gets passed to the compiler as an internal data structure?

Why is the reader important as a separate function, idea, phase? Why doesn't the compiler serialize those reader forms and convert it to data structures internally?

Another question is: It is possible to write a program that would generate data structures directly and so they could then be directly passed to the compiler (without stepping through the reader stage), neither through a macro coded in a source file. How can try to do this?

Very nice explation: What are the tasks of the "reader" during Lisp interpretation?

noisesmith noisesmith · Accepted Answer · 2014-05-27T20:59:26

The compiler does not receive characters as input, it receives structured Clojure data. The compiler resolves symbols to vars (by finding them in the appropriate namespace).

Let's look at what your read-string call actually returns:

user> (def input (read-string "(+ 1 2 3)"))
#'user/input
user> input
(+ 1 2 3)
user> (clojure.pprint/pprint (map (juxt identity type) input))
([+ clojure.lang.Symbol]
 [1 java.lang.Long]
 [2 java.lang.Long]
 [3 java.lang.Long])
nil

As you can see, there are no characters being sent to the compiler - the reader reduces everything to a primitive Clojure type (more or less the types supported by edn data), and the compiler turns this into runable code.

Regarding "generating these datastructures directly", this is exactly what macros do (though at least the macro invocation will be handled by the reader for a macro call).

user> (defmacro construct-addition [n] (list + 1 2 n))
#'user/construct-addition
user> (construct-addition 5)
8
user> (macroexpand '(construct-addition 5))
(#<core$_PLUS_ clojure.core$_PLUS_@190767cf> 1 2 5)

Clojure evaluation process, the reader's job and directly passed data-structures to clojures compiler/evaluator

1 Answers