1
votes

The Eigen web site says:

Explicit vectorization is performed for SSE 2/3/4, AVX, FMA, AVX512, ARM NEON (32-bit and 64-bit), PowerPC AltiVec/VSX (32-bit and 64-bit) instruction sets, and now S390x SIMD (ZVector) with graceful fallback to non-vectorized code.

Does that mean if you compile, e.g. with FMA, and the CPU you are running on does not support it it will fall back to completely unvectorised code? Or will it fall back to the best vectorisation available?

If not, is there any way to have Eigen compile for all or several SIMD ISAs and automatically pick the best at runtime?

Edit: To be clear I'm talking about run-time fallbacks.

1
Given that the linked FAQ says, "With SSE, at least SSE2 is required. SSE3, SSSE3 and SSE4 are optional, and will automatically be used if they are enabled.", I suspect that it will fall back to the best vectorization option available. But there will still be hard limitations. For example, you can't vectorize double-precision floating-point operations without at least SSE2. SSE supports only single-precision. I assume that's the reason behind the requirement in the quotation. - Cody Gray♦
You can see this by digging into the source code. In the linked function, the version that has been optimized for SSE, you can see that there is an implementation for SSE3, but when that's not available, it falls back to a version that requires only SSE2. This is chosen with preprocessor macros, so you need to make sure those are correctly defined. - Cody Gray♦
Those seem to be compile-time preprocessor directives. I'm talking about run-time fallbacks. - Timmmm
I see absolutely no indication, whether in the code or in the documentation, that Eigen supports any type of run-time dispatching/delegation. Where did you get the idea that it does? (Note, however, that some compilers support these features, like Intel's C compiler. So, since this is a header-only library, you could compile it using one of these compilers and possibly get run-time dispatching.) - Cody Gray♦
From the quote that I posted. It is ambiguous if it is talking about run-time or compile-time. This is the whole point of my question!! - Timmmm

1 Answers

3
votes

There is absolutely no run-time dispatching in Eigen. Everything that happens happens at compile-time. This is where you need to make all of the choices, either via preprocessor macros that control the library's behavior, or using optimization settings in your C++ compiler.

In order to implement run-time dispatching, you would either need to check and see what CPU features were supported on each and every call into the library and branch into the applicable implementation, or you would need to do this check once at launch and set up a table of function pointers (or some other method to facilitate dynamic dispatching). Eigen can't do the latter because it is a header-only library (just a collection of types and functions), with no "main" function that gets called upon initialization where all of this setup code could be localized. So the only option would be the former, which would result in a significant performance penalty. The whole point of this library is speed; introducing this type of performance penalty to each of the library's functions would be a disaster.

The documentation also contains a breakdown of how Eigen works, using a simple example. This page says:

The goal of this page is to understand how Eigen compiles it, assuming that SSE2 vectorization is enabled (GCC option -msse2).

which lends further credence to the claim that static, compile-time options determine how the library will work.

Whichever instruction set you choose to target, the generated code will have those instructions in it. If you try to execute that code on a processor that does not support these instructions (for example, you compile Eigen with AVX optimizations enabled, but you run it on an Intel Nehalem processor that doesn't support the AVX instruction set), then you will get an invalid instruction exception (presented to your program in whatever form the operating system passes CPU exceptions through). This will happen as soon as your CPU encounters an unrecognized/unsupported instruction, which will probably be deep in the bowels of one of the Eigen functions (i.e., not immediately upon startup).

However, as I said in the comments, there is a fallback mechanism of sorts, but it is a static one, all done at compile time. As the documentation indicates, Eigen supports multiple vector instruction sets. If you choose a lowest-common-denominator instruction set, like SSE2, you will still get some degree of vectorization. For example, although SSE3 may provide a specialized instruction for some particular task, if you can't target SSE3, all hope is not lost. There is, in many places, code that uses a series of SSE2 instructions to accomplish the same task. These are probably not as efficient as if SSE3 were available, but they'll still be faster than having no vector code whatsoever. You can see examples of this by digging into the source code, specifically the arch folder that contains the tidbits that have been specifically optimized for the different instruction sets (often by the use of intrinsics). In other words, you get the best code it can give you for your target architecture/instruction set.