The use of macros like DEFER
, and complicated C macrology in general, depends on understanding how the C preprocessor actually expands macro expressions. It doesn't just attempt to reduce all expression trees the way a conventional programming language does, but rather, it works on a linear token stream, and has an implicit "cursor" at the point where it's currently examining the stream for possible replacements. Within any given "stack frame" of the expansion process, the cursor never moves backwards, and once a token has been passed in the stream it is not examined again.
Walking through the first example of DEFER
's operation:
DEFER(A)() // cursor starts at the head of the sequence
^ // identifies call to DEFER - push current position
DEFER( A )() // attempt to expand the argument (nothing to do)
^
// replace occurrences of id in DEFER with A,
// then replace the call to it with the substituted body
A EMPTY() () // pop cursor position (start of pasted subsequence)
^ // doesn't find an expansion for A, move on
A EMPTY() () // move to next token
^ // EMPTY() is a valid expansion
A () // replace EMPTY() with its body in the same way
^ // continuing...
A () // `(` is not a macro, move on
^
A ( ) // `)` is not a macro, move on
^
A () // end of sequence, no more expansions
^
The cursor moved past A
during the "rescan" of DEFER
's body, after the arguments had been substituted, which is the second and last attempt to expand that set of tokens. Once the cursor has moved past A
it does not return to it during that expansion sequence, and since the "rescan" is at the top level there is no following expansion sequence.
Now consider the same expression, but wrapped in a call to EXPAND
:
EXPAND(DEFER(A)()) // cursor starts at the head etc.
^ // identifies call to EXPAND
EXPAND( DEFER(A)() ) // attempt to expand the argument
^ // this does the same as the first
// example, in a NESTED CONTEXT
// replace occurrences of __VA_ARGS__ in EXPAND with A ()
// then replace the call with the substituted body
A () // pop cursor position (start of pasted subsequence)
^ // identifies A, and can expand it this time
Because argument lists are expanded in a stacked context, and the cursor position is restored to the position in front of the original call for the rescan pass, placing a macro invocation in any macro's argument list - even one that actually does nothing, like EXPAND
- gives it a "free" extra run over by the expansion cursor, by resetting the cursor's position in the stream an extra time for an extra rescan pass, and therefore giving each freshly constructed call expression (i.e. the pushing together of a macro name and a parenthesized argument list) an extra chance at being recognised by the expander. So all EVAL
does is give you 363 (3^5+3^4+3^3+3^2+3, someone check my math) free rescan passes.
So, addressing the questions in light of this:
- "painting blue" doesn't work quite like that (the explanation in the wiki is a bit misleadingly phrased, although it's not wrong). The name of a macro, if generated within that macro, will be painted blue permanently (C11 6.10.3.4 "[blue] tokens are no longer available for further replacement even if they are later (re)examined"). The point of
DEFER
is rather to ensure that the recursive invocation doesn't get generated on the macro's expansion pass, but instead is ...deferred... until an outer rescan step, at which point it won't get painted blue because we're no longer within that named macro. This is why REPEAT_INDIRECT
is function-like; so that it can be prevented from expanding into anything mentioning the name of REPEAT
, as long as we're still within the body of REPEAT
. It requires at least one further free pass after the originating REPEAT
completes to expand away the spacing EMPTY
tokens.
- yes,
EXPAND
forces an additional expansion pass. Any macro invocation grants one extra expansion pass to whatever expression was passed in its argument list.
- the idea of
DEFER
is that you don't pass it a whole expression, just the "function" part; it inserts a spacer between the function and its argument list that costs one expansion pass to remove.
- therefore the difference between
EXPAND
and DEFER
is that DEFER
imposes the need for an extra pass, before the function you pass it gets expanded; and EXPAND
provides that extra pass. So they are each other's inverse (and applied together, as in the first example, are equivalent to a call using neither).
- yes,
OBSTRUCT
costs two expansion passes before the function you pass it gets expanded. It does this by DEFER
ing the expansion of the EMPTY()
spacer by one (EMPTY EMPTY() ()
), burning the first cursor reset on getting rid of the nested EMPTY
.
OBSTRUCT
is needed around REPEAT_INDIRECT
because WHEN
expands (on true) to a call to EXPAND
, which will burn away one layer of indirection while we're still within the call to REPEAT
. If there was only one layer (a DEFER
), the nested REPEAT
would be generated while we're still in REPEAT
's context, causing it to be painted blue and killing the recursion right there. Using two layers with OBSTRUCT
means that the nested REPEAT
won't be generated until we reach the rescan pass of whatever macro call is outside REPEAT
, at which point we can safely generate the name again without it being painted blue, because it's been popped from the no-expand stack.
So, this method of recursion works by using an external source of a large number of rescan passes (EVAL
), and deferring the expansion of a macro's name within itself by at least one rescan pass so it happens further down the call stack. The first expansion of the body of REPEAT
happens during the rescan of EVAL[363]
, the recursive invocation is deferred until the rescan of EVAL[362]
, the nested expansion deferred... and so on. It's not true recursion, as it can't form an infinite loop, but instead relies on an external source of stack frames to burn through.
inline
. Things you can do better with macros: generate enumerations whose value can be converted to/from strings; define terse serialization helpers; avoid code repetition such asstd::forward<decltype(something)>(something)
toFWD(something)
; safe wrappers for things likeoffsetof
(I created a wrapper that throws a compilation error if the passed type is not standard-layout); better assertions; no-boilerplate unit tests; and much more. Macros have their uses. – Vittorio Romeo()
it doesn't look back to see if there was a macro name preceding this invocation. That's what makesDEFER
work. Step #1 (fully replace each argument in isolation) makesEXPAND
work. – Igor Tandetnik