Using a nightly TensorFlow build for training with Cloud ML Engine

3
votes

If I need to use a nightly TensorFlow build in a Cloud ML Engine training job, how do I do it?

1
google-cloud-mlgoogle-cloud-ml-engine

1 Answers

5
votes

  • Download a nightly build from https://github.com/tensorflow/tensorflow#installation. How to pick the right build:

    • use "Linux CPU-only" or "Linux GPU" depending on whether you need to use GPUs for training,
    • use the Python 2 build.

  • Rename the .whl file, for example 

    mv tensorflow-1.0.1-cp27-cp27mu-linux_x86_64.whl \
   tensorflow-1.0.1-cp27-none-linux_x86_64.whl

    (here we renamed the cpu27mu to none. Pip parses that part to detect whether a .whl package is suitable for a platform, but that particular name doesn't work on some older versions of pip)

  • Upload the .whl file to GCS and specify it as one of the package_uris when submitting a Cloud ML Engine training job.

Note that instead of using a nightly build you can also build TensorFlow from source as described in https://www.tensorflow.org/install/install_sources.