Optimization barrier for microbenchmarks in MSVC: tell the optimizer you clobber memory?

Question

Chandler Carruth introduced two functions in his CppCon2015 talk that can be used to do some fine-grained inhibition of the optimizer. They are useful to write micro-benchmarks that the optimizer won't simply nuke into meaninglessness.

void clobber() {
  asm volatile("" : : : "memory");
}

void escape(void* p) {
  asm volatile("" : : "g"(p) : "memory");
}

These use inline assembly statements to change the assumptions of the optimizer.

The assembly statement in clobber states that the assembly code in it can read and write anywhere in memory. The actual assembly code is empty, but the optimizer won't look into it because it's asm volatile. It believes it when we tell it the code might read and write everywhere in memory. This effectively prevents the optimizer from reordering or discarding memory writes prior to the call to clobber, and forces memory reads after the call to clobber†.

The one in escape, additionally makes the pointer p visible to the assembly block. Again, because the optimizer won't look into the actual inline assembly code that code can be empty, and the optimizer will still assume that the block uses the address pointed by the pointer p. This effectively forces whatever p points to be in memory and not not in a register, because the assembly block might perform a read from that address.

(This is important because the clobber function won't force reads nor writes for anything that the compilers decides to put in a register, since the assembly statement in clobber doesn't state that anything in particular must be visible to the assembly.)

All of this happens without any additional code being generated directly by these "barriers". They are purely compile-time artifacts.

These use language extensions supported in GCC and in Clang, though. Is there a way to have similar behaviour when using MSVC?

† To understand why the optimizer has to think this way, imagine if the assembly block were a loop adding 1 to every byte in memory.

It looks like _ReadWriteBarrier might be the answer for clobber. I don't know about escape though. Maybe _ReadWriteBarrier plus handing off the pointer to some externally defined function. — user786653
Oh, I forgot to mention another feature of these: they generate no code whatsoever. Any effect they have disappears after the optimizer is done. Nothing sticks until runtime. They're purely compile-time. — R. Martinho Fernandes
Like @user786653 said, _ReadWriteBarrier (or perhaps just _ReadBarrier/_WriteBarrier if that's all that is needed) will have the same effect in MSVC as clobber. For escape, my experience in analyzing assembly output is that MSVC will do the right thing if you just mark the variable volatile. Of course, there is some runtime overhead in that, because the generated code will always keep the variable updated in memory. It's not a perfect solution, but I haven't found anything better. — Cody Gray
@R.MartinhoFernandes ... as long as you are using this mechanism for benchmarking, but not relying on it to prevent migration of reads/writes for threaded code, I quiesce. — David Thomas
Have you researched std::atomic_signal_fence and atomic_thread_fence? — David Thomas

Leon Leon · Accepted Answer · 2016-07-08T16:01:40

Given your approximation of escape(), you should also be fine with the following approximation of clobber() (note that this is a draft idea, deferring some of the solution to the implementation of the function nextLocationToClobber()):

// always returns false, but in an undeducible way
bool isClobberingEnabled();

// The challenge is to implement this function in a way,
// that will make even the smartest optimizer believe that
// it can deliver a valid pointer pointing anywhere in the heap,
// stack or the static memory.
volatile char* nextLocationToClobber();

const bool clobberingIsEnabled = isClobberingEnabled();
volatile char* clobberingPtr;

inline void clobber() {
    if ( clobberingIsEnabled ) {
        // This will never be executed, but the compiler
        // cannot know about it.
        clobberingPtr = nextLocationToClobber();
        *clobberingPtr = *clobberingPtr;
    }
}

UPDATE

Question: How would you ensure that isClobberingEnabled returns false "in an undeducible way"? Certainly it would be trivial to place the definition in another translation unit, but the minute you enable LTCG, that strategy is defeated. What did you have in mind?

Answer: We can take advantage of a hard-to-prove property from the number theory, for example, Fermat's Last Theorem:

bool undeducible_false() {
    // It took mathematicians more than 3 centuries to prove Fermat's
    // last theorem in its most general form. Hardly that knowledge
    // has been put into compilers (or the compiler will try hard
    // enough to check all one million possible combinations below).

    // Caveat: avoid integer overflow (Fermat's theorem
    //         doesn't hold for modulo arithmetic)
    std::uint32_t a = std::clock() % 100 + 1;
    std::uint32_t b = std::rand() % 100 + 1;
    std::uint32_t c = reinterpret_cast<std::uintptr_t>(&a) % 100 + 1;

    return a*a*a + b*b*b == c*c*c;
}

Optimization barrier for microbenchmarks in MSVC: tell the optimizer you clobber memory?

2 Answers