Common Lisp Double-Backquote, Unquote, Quote, Unquote sequence?

Question

I'm reading Let Over Lambda, which deals with some pretty deeply layered macro authoring. It's fascinating and I'm mostly managing to keep up with it.

In Chapter 4 Hoyte implements reader macros for CL-PPCRE match and replace functions, such that you can do things like:

(#~m/(foo|bar)\d+/ "Some foo99")    ; matches!
(#~s/foo(\d+)/bar\1/, "Some foo99") ; "Some bar99

In order to achieve this, we define a macro that uses the double-backquote, since it is actually expanded by a wrapper macro, which needs the quoted value (it returns a lambda form). Within the quasi-quoted list, there is some use of the following sequence ,',varname, which I can't get my head around. What does the initial ,' do here?

(defmacro! pcre/match-lambda-form (o!args)
  "Expands to a lambda that applies CL-PPCRE:SCAN"
  ``(lambda (,',g!str)
      (cl-ppcre:scan ,(car ,g!args)
                     ,',g!str)))

Actually, it's probably better that I distill that down to something that uses just defmacro, for clarity if you haven't read the book. str is a symbol and args is a list:

(defmacro pcre/match-lambda-form (args)
  "Expands to a lambda that applies CL-PPCRE:SCAN"
  ``(lambda (,',str)
      (cl-ppcre:scan ,(car ,args)
                     ,',str)))

Are the quotes basically double-quoting the inner parts, so that the result can be unquoted twice? Effectively putting 'str into the expanded form, instead of just str?

EDIT | Thanks to Terje D. and some playing around in the REPL, this is pretty much the situation:

(defvar a 42)

(equal ``(,,a)  '(list 42)) ; T
(equal ``(,a)   '(list a))  ; T
(equal ``(,',a) ''(42))     ; T
(equal ``(a)    ''(a))      ; T (obviously)

So:

Doubly-unquoted, form is fully expanded.
Singly-unquoted, form is not expanded.
Unquoted with a comma, form is fully expanded and the result quoted.

your last defmacro references str. What is it? Where is it defined? Can you show how pcre/match-lambda-form is called? — Will Ness
Argh, this is going to take some explaining as there are many layers of macro going on in reality. This book is pushing boundaries with macro authoring (for fun, and for education). str is a symbol that has been defined by an enclosing LET and represents a variable name that is safe to use. — d11wtq
pcre/match-lambda-form is called like so: (pcre/match-lambda-form '("foo")), where the "foo" has been read by a stream reader processing strings of the form #~m/foo/. — d11wtq
I'm reading it in print form, but it's (partially) available online too: letoverlambda.com/index.cl/toc — d11wtq
There is no right or wrong here. According to the Hyperspec, section 2.4.6 (Backquote): An implementation is free to interpret a backquoted form F1 as any form F2 that, when evaluated, will produce a result that is the same under equal as the result implied by the above definition, provided that the side-effect behavior of the substitute form F2 is also consistent with the description given above. I.e the forms must be fully evaluated before any comparison is done. — Terje D.

Terje D. Terje D. · Accepted Answer · 2013-07-03T09:09:16

During evaluation of a doubly backquoted form, the inner backquote is handled first, and the result is a singly backquoted form. During evaluation of the inner backquoted form, only elements preceeded by two commas are evaluated. However, the result of evaluating these doubly unquoted elements are still (singly) unquoted, and are thus evaluated again when the resulting singly backquoted form are evaluated. To achieve evaluation only in the inner backquoted form, an ordinary quote has to be inserted, resulting in ,',.

See how

(let ((tmp (gensym)))
    ``(lambda (,tmp ,,tmp ,',tmp) ()))

evaluates to

`(LAMBDA (,TMP ,#:G42 #:G42) nil)

Common Lisp Double-Backquote, Unquote, Quote, Unquote sequence?

2 Answers