I’m working on an object detector using Cloud Machine Learning Engine via a Cloud-VM instance. Following the tutorial (https://cloud.google.com/blog/big-data/2017/06/training-an-object-detector-using-cloud-machine-learning-engine).
I get a module import error on Google Cloud Platform when I submit the training job below:
gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%s` \
--job-dir=${YOUR_GCS_BUCKET}/train \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz \
--module-name object_detection.train \
--region us-central1 \
--config object_detection/samples/cloud/cloud.yml \
-- \
--train_dir=${YOUR_GCS_BUCKET}/train \
--pipeline_config_path=${YOUR_GCS_BUCKET}/data/faster_rcnn_resnet101_coco.config
The error is below:
...object_detection/utils/visualization_utils.py", line 24, in <module>
import matplotlib.pyplot as plt
ImportError: No module named matplotlib.pyplot
I've installed matplotlib using pip install. This code works fine python2.7 -c 'import matplotlib.pyplot as plt'.
The matplotlib error is resolved by adding package names in the REQUIRED_PACKAGES list inside the setup.py program file.
Also, Please see my setup.py file..
"""Setup script for object_detection."""
from setuptools import find_packages
from setuptools import setup
import subprocess
subprocess.check_call(['apt-get', 'update'])
subprocess.check_call(['apt-get', 'install', 'python-tk'])
REQUIRED_PACKAGES = ['Pillow>=1.0', 'matplotlib']
setup(
name='object_detection',
version='0.1',
install_requires=REQUIRED_PACKAGES,
include_package_data=True,
packages=[p for p in find_packages() if p.startswith('object_detection')],
description='Tensorflow Object Detection Library',
)
But, even after resolving this, there are some other errors coming in this scenario as matplotlib has a dependency on python-tk package.
ps-replica-0 Could not find a version that satisfies the requirement python-tk (from object-detection==0.1) (from versions: ) ps-replica-0
ps-replica-0 No matching distribution found for python-tk (from object-detection==0.1) ps-replica-0
ps-replica-0 Command '['pip', 'install', '--user', u'object_detection-0.1.tar.gz']' returned non-zero exit status 1 ps-replica-0
ps-replica-0 Module completed; cleaning up. ps-replica-0
But python-tk/python3-tk is not available in the pip package. In order to do so, we need to do sudo apt-get install python-tk or sudo apt-get install python3-tk
Google Cloud-ML runs python 2.7. Hence, we need install python-tk before running out tensorflow training program.
Now, can someone please help me in order to command Cloud ML to install python-tk using apt-get before running tensorflow.
Update_01:*
I'm getting another set of errors. It seems it's caused by the failure of python setup.py egg_info. Also, there's this..
Command '['apt-get', 'install', 'python-tk']' returned non-zero exit status 1
The error log is shown below. Thanks in advance for your help.
ps-replica-2
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-BhSDtP-build/
ps-replica-2
Command '['pip', 'install', '--user', '--upgrade', '--force-reinstall', '--no-deps', u'object_detection-0.1.tar.gz']' returned non-zero exit status 1
The replica ps 0 exited with a non-zero status of 1. Termination reason: Error.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-C3hdCp-build/setup.py", line 8, in <module>
subprocess.check_call(['apt-get', 'install', 'python-tk'])
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['apt-get', 'install', 'python-tk']' returned non-zero exit status 1
The replica ps 1 exited with a non-zero status of 1. Termination reason: Error.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-iR0TqP-build/setup.py", line 8, in <module>
subprocess.check_call(['apt-get', 'install', 'python-tk'])
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['apt-get', 'install', 'python-tk']' returned non-zero exit status 1
The replica ps 2 exited with a non-zero status of 1. Termination reason: Error.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-BhSDtP-build/setup.py", line 8, in <module>
subprocess.check_call(['apt-get', 'install', 'python-tk'])
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['apt-get', 'install', 'python-tk']' returned non-zero exit status 1
To find out more about why your job exited please check the logs: https://console.cloud.google.com/logs/viewer?project=640992742297&resource=ml_job%2Fjob_id%2Froot_object_detection_1510462119&advancedFilter=resource.type%3D%22ml_job%22%0Aresource.labels.job_id%3D%22root_object_detection_1510462119%22"
The job submission code:
gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%s` \
--job-dir=${YOUR_GCS_BUCKET}/train \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz \
--module-name object_detection.train \
--config object_detection/samples/cloud/cloud.yml \
-- \
--train_dir=${YOUR_GCS_BUCKET}/train \
--pipeline_config_path=${YOUR_GCS_BUCKET}/data/faster_rcnn_resnet101_coco.config
Thanks in advance..
Update_02: Solution
Thanks to @Dennis Liu for the solution. No need to install python-tk package.
Apart from that there will be one more error, which can be solved by changing tf.train.get_or_create_global_step() to tf.contrib.framework.get_or_create_global_step() at line 103 in object_detection/builders/optimizer_builder.py. Solution Link
