Re-build Tensorflow with desired optimization flags

Question

and thanks in advance for your consideration,

I just installed tensorflow (on a new machine with Ubuntu 16.04 and CUDA 8.0 already installed) using the following procedure:

Initially, I used --copt=-march=native. I received the message

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

So in an attempt to fix this, I searched for solutions and used the answer to the following How to compile Tensorflow with SSE4.2 and AVX instructions?

by using the above procedure from nVidia, starting from

bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda -k //tensorflow/tools/pip_package:build_pip_package

but am still receiving the same messages as above. I feel as though I'm making a very simple error, any ideas?

Thanks!

If you configured and built properly with the same optimization flags and are still seeing these warnings it might be your CPU architecture does not support those SIMD instructions. Use gcc -march=native -Q --help=target to see which ones are available — Adriano
Thanks Adriano. If I'm not mistaken, the instructions the warnings refer to (mavx, mavx2, mfma, msse, msse2, msse3, msse4, msse4.1, msse4.2) are supported and enabled (pastebin.com/U4S18aRQ). — Chaztikov

Adriano Adriano · Accepted Answer · 2017-04-11T20:06:53

By following the NVIDIA instructions you are resetting the TensorFlow repository to an older commit, before the SIMD instructions optimization was made available (1.0r):

git reset --hard 70de76e

This commit dates back to a previous release when this feature was not yet implemented, so it is actually working as it's supposed to.

The solution is to follow the official TensorFlow documentation.

For future situations, it is always recommended to use official resources before reaching out for third party solutions, as more they may be helpful, the official ones are more reliable and better maintained.

Notice during configure you're not prompted which CPU instructions you want to build TF with as it should due to the reason above, and therefore, you're unable to build with them.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Follow the official docs accordingly and it will work. If you have any follow up questions feel free to ask or if you face any problems open an issue on Github :)

Re-build Tensorflow with desired optimization flags

1 Answers