How to correctly determine -march and -mtune for Intel processors?

Question

I am currently building a software from source that is performance critical for me. Therefore I want to optimize it for running on my specific Intel CPUs. The building process requires that I set the -march and -mtune flag.

If on my processor node i use

gcc -march=native -Q --help=target|grep march
gcc -mtune=native -Q --help=target|grep mtune

I get "core-avx2" for march and "generic" for mtune. However with

cat /proc/cpuinfo

I get:

processor   : 23
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
stepping    : 2
microcode   : 0x3d
cpu MHz     : 2599.993
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 13
cpu cores   : 12
apicid      : 58
initial apicid  : 58
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts
bogomips    : 4599.35
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

By going to the homepage of Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz (https://ark.intel.com/content/www/de/de/ark/products/81709/intel-xeon-processor-e5-2670-v3-30m-cache-2-30-ghz.html) I found out: Codename -> Products formerly haswell

If I use

gcc -march=haswell -Q --help=target|grep march
gcc -mtune=haswell -Q --help=target|grep mtune

I get "haswell" for both. So shouldn't I actually use haswell as march instead of core-avx2? What is the best possible choice?

Btw, I am using GCC 4.8.5 on CentOS7.

Thanks!

Edit:

gcc -march=native -Q --help=target | grep -- '-march=' | cut -f3

-> core-avx2

gcc -mtune=native -Q --help=target | grep -- '-mtune=' | cut -f3

-> generic

Answer 1

In the gcc version you're using, Haswell was called core-avx2. Other microarchitectures had also crappy names. For example, Ivy Bridge, Sandy Bridge, and Westmere were called, core-avx-i, corei7-avx, and corei7, respectively. Starting with gcc 4.9.0, the actual names of the microarchitectures are used, so gcc will print Haswell when using gcc -march=native -Q --help=target|grep march on a Haswell processor instead of core-avx2 (see the patch).

When passing -mtune=native to gcc and the host processor is not known to the version of gcc you're using, it will apply generic tuning. Your processor model (63) is only known to gcc 5.1.0 and later (see the patch).

The name-printing part of -Q --help=target has to pick some name for -march=native. For CPUs too new for your GCC to recognize specifically, it will pick something like Broadwell if the processor supports ADX, or the microarchitecture that supports the highest SIMD extension (up to AVX2) that is supported on the host processor (as determined by cpuid).

But the actual effect of -march=native is to enable all the appropriate -mavx -mpopcnt -mbmi2 -mcx16 and so on options, all detected separately using cpuid. So for code-gen purposes, -march=native always works for enabling ISA extensions that your GCC knows how to use, even if it doesn't recognize your CPU.

But for setting tune options, -march=native or -mtune=native totally fails and falls back to generic when it doesn't recognize your CPU exactly. It unfortunately doesn't do things like tune=intel for unknown-Intel CPUs.

---

On your processor, gcc knows that it supports AVX2, so it assumes that it is a Haswell processor (called core-avx2 in your gcc version) because AVX2 is supported starting on Haswell, but it doesn't know for sure that it is actually a Haswell processor. That's why it applies generic tuning instead of tuning for core-avx2 (i.e., Haswell). But in this case, I think this would have the same effect as tuning for core-avx2 because, to that compiler version, only Haswell supports AVX2 and the compiler knows that the host processor supports AVX2. In general, though, it may not tune for the native microarchitecture even if -march was guessed correctly on an unkown CPU.

(Editor's note: no, tune=generic doesn't adapt to which instruction-set options are enabled. It's still fully generic tuning, including caring about CPUs like AMD Phenom or Intel Sandybridge that don't support AVX2. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80568 and Why doesn't gcc resolve _mm256_loadu_pd as single vmovupd?.

This is one reason why you should use -march=native or -march=haswell (with a new enough gcc), not just -mavx2 -mfma. The other reason is that you're probably going to forget -mbmi2 -mpopcnt -mcx16, and maybe even forget -mfma)

Answer 2

Btw, I am using GCC 4.8.5 on CentOS7.

If performance is critical, you should use a more recent version of GCC. The 4.8 release series dates back to 2013, and lacks many performance enhancements that are present in current versions. Current versions have significantly expanded tuning options for x86, including -march settings for many processor families that didn't exist in 2013.