Preprocessor inconsistencies in Visual Studio

Question

I'm compiling the following C++ code in Visual Studio 2015 (update 3):

#include <iostream>
using namespace std;

////////////////////////////////////////

#define UNDERSCORE1(a,b) a ## _ ## b
#define UNDERSCORE(a,b) UNDERSCORE1(a,b)

#define STRINGIFY1(x) #x
#define STRINGIFY(x) STRINGIFY1(x)

#define VALUE(x) UNDERSCORE(x, VALUE)
#define NEXT(x) (VALUE(x) + 1)

/////////////////////////////////////////

#define X1_VALUE 0
#define X2_VALUE NEXT(X1)
#define X3_VALUE NEXT(X2)
#define TOTAL NEXT(X3)

int main() {
    cout << STRINGIFY(TOTAL) << endl;
    cout << TOTAL << endl;
    return 0;
}

The result printed to stdout is very strange:

(X3_VALUE + 1)
3

When trying the same on gcc, build fails (expectedly).
When commenting out cout << TOTAL << endl; I get something different altogether:

(NEXT(X2) + 1)

Actually gcc behavior makes sense, since NEXT macro is called recursively: NEXT(X3) is expanded to X3_VALUE which in turn expands to NEXT(X2), so the second expansion of NEXT macro (NEXT(X2)) is not performed.

What doesn't make sense is Visual Studio behavior:

When printing the macro TOTAL using STRINGIFY, NEXT seems to be expanded twice to yield X3_VALUE.
When compiling the macro TOTAL directly to send it to cout, NEXT is expanded all the way! As if the preprocessor ran multiple times to recursively expand NEXT.

Another thing I tried was to compile this code in Visual Studio with the /P compiler option, to get the preprocessed code:

int main() {
    cout << "(X3_VALUE + 1)" << endl;
    cout << (((0 + 1) + 1) + 1) << endl;
    return 0;
}

So is it, as I suspect, a bug in Visual Studio preprocessor? Or a legit undefined behavior?
Perhaps this behavior can be abused to truly expand macros recursively? I know that a limited recursion is possible with some tricks but is limited to a predefined number of scans. In this case I did not observe a limit to the number of times NEXT is expanded.

This probably isn't the problem, but names that begin with an underscore followed by a capital letter (_UNDERSCORE, _STRINGIFY) and names that contain two consecutive underscores are reserved for the implementation. Don't use them in your code. — Pete Becker
I would start by avoiding reserved identifiers. _ followed by an uppercase letter is reserved to the implementation for any use. — StoryTeller - Unslander Monica
@PeteBecker I'll change the question to avoid underscore followed by a capital letter. The problem remains the same. — Amir Gonnen
It is behavior #6. This goes back to about 1983, long before C89 nailed down the expected behavior. Early K&R preprocessors were inconsistent about it, they gambled wrong. Possibly helped by it having to be able to run in 96KB of RAM. They decided to not break the codebase of existing customers, a notorious affliction Microsoft suffers from, that has been changing of late. Tagging a question like this with [visual-c++] is usually a good way to avoid irrelevant comments. — Hans Passant

Mac Mac · Accepted Answer · 2018-11-23T06:41:20

As Hans mentioned in the comments, the MSVC preprocessor is not conformant.

You can enable the conformant preprocessor with -experimental:preprocessor.

Here is a simplified repro + solution: https://godbolt.org/z/7u_-bH

Preprocessor inconsistencies in Visual Studio

1 Answers