Failed building wheel in Google ML engine

Question

I follow this tutorial to submit machine learning job to Google ML engine. Then I faced the error ImportError: No module named matplotlib.pyplot but solved it by adding the Matplotlib into RequiredPackages in setup.py. Then I face another error which is ImportError: No module named _tkinter, please install the python-tk package. I found this solution and this solution but do not help and give me another error.

Failed building wheel for my-package

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-T6kjZl-build/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-4LpzWh-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-T6kjZl-build/

Command '['pip', 'install', '--user', '--upgrade', '--force-reinstall', '--no-deps', u'my-package-0.1.1.tar.gz']' returned non-zero exit status 1

My setup.py.

"""Setup script for object_detection."""
import logging
import subprocess
from setuptools import find_packages
from setuptools import setup
from setuptools.command.install import install

class CustomCommands(install):
    def RunCustomCommand(self, command_list):
        p = subprocess.Popen(
        command_list,
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT)
        # Can use communicate(input='y\n'.encode()) if the command run requires
        # some confirmation.
        stdout_data, _ = p.communicate()
        logging.info('Log command output: %s', stdout_data)
        if p.returncode != 0:
            raise RuntimeError('Command %s failed: exit code: %s', (command_list, p.returncode))
        
    def run(self):
        self.RunCustomCommand(['apt-get', 'update'])
        self.RunCustomCommand(
        ['apt-get', 'install', '-y', 'python-tk'])
        install.run(self)
    
REQUIRED_PACKAGES = ['Pillow>=1.0', 'Matplotlib>=2.1']
setup(
name='object_detection',
version='0.1',
install_requires=REQUIRED_PACKAGES,
include_package_data=True,
packages=[p for p in find_packages() if
p.startswith('object_detection')],
description='Tensorflow Object Detection Library',
cmdclass={
    'install': CustomCommands,
})

Do you mind copying your full setup.py and also possibly providing more of the logs? — rhaertel80
I have added the setup.py. Actually I have posted all the log. — SamTew
I copied your setup.py with one minor change: I set package=find_packages(). Then I created a simple main.py: import matplotlib.pyplot import _tkinter print "Hello, world!" Submitted the job, and it worked. gcloud ml-engine jobs submit training matplotlib --package-path=a --module-name="a.main" --staging-bucket=gs://my-bucket — rhaertel80
I tried walking through the blogpost. Lots of minor bugs, that I'll submit patches for. I tried using your setup.py file verbatim, and it seems to work, that is, I don't get the same error as you are seeing. That said, I still get an error about a missing method. I will try to fix and patch that as well. — rhaertel80

Vaibhav Sahu Vaibhav Sahu · Accepted Answer · 2018-11-04T14:52:59

The tutorial blog link that you are following is outdated I think. I followed this one you need to change runtime to 1.9 instead of 1.8 in cloud ml training command

From tensorflow/models/research/
gcloud ml-engine jobs submit training `whoami`_object_detection_pets_`date +%m_%d_%Y_%H_%M_%S` \
    --runtime-version 1.9 \
    --job-dir=gs://${YOUR_GCS_BUCKET}/model_dir \
    --packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
    --module-name object_detection.model_main \
    --region us-central1 \
    --config object_detection/samples/cloud/cloud.yml \
    -- \
    --model_dir=gs://${YOUR_GCS_BUCKET}/model_dir \
    --pipeline_config_path=gs://${YOUR_GCS_BUCKET}/data/faster_rcnn_resnet101_pets.config

after changing runtime everything worked perfect. No need to change any setup file. No Library errors.

Failed building wheel in Google ML engine

1 Answers