ImportError running Google's python appengine on Ubuntu

Question

I'm trying to teach myself python using Google's AppEngine, and I can't get the dev server running. I get this error:

Traceback (most recent call last):
File "/opt/google_appengine/google_appengine_1.2.7/dev_appserver.py", line 60, in run_file(file, globals()) File "/opt/google_appengine/google_appengine_1.2.7/dev_appserver.py", line 57, in run_file execfile(script_path, globals_) File "/opt/google_appengine/google_appengine_1.2.7/google/appengine/tools/dev_appserver_main.py", line 65, in from google.appengine.tools import os_compat ImportError: cannot import name os_compat

Ubuntu 9.10 comes with python2.6 (didn't work), and I installed python2.5 (didn't work), and have tried running it with python dev_appserver.py helloWorld (didn't work) as well as running dev_appserver.py after editing the first line to be:

#!/usr/bin/env python2.5

I can't seem to find anything online with this error. The only problem I've found is about using python 2.5, and I think I've solved that.

Kyle suggested I need to set my PYTHONPATH variable. After running

export PYTHONPATH=/opt/google_appengine/google_appengine_1.2.7

I still get the same error trying to run dev_appserver.py. Am I setting PYTHONPATH wrong? Alternatively, how do I uninstall the protocol buffers python project? I have no use for Ubuntu One and had already uninstalled it.

Answer 1

The problem appears to be the fact that Karmic Koala 9.10 (the latest version of Ubuntu) ships with Ubuntu One, a python app that depends on Google's protocol buffers library. The python-protobuf package provides the google.protobuf package in /usr/lib/pymodules/python2.6.

Unfortunately, the AppEngine SDK includes another package called google.appengine. So somewhere in your code, the google package is being imported, and the package that contains protobuf is being found on PYTHONPATH first. Python caches the first package it finds in sys.modules, so the second google package in the SDK will never be imported.

You could move the google AppEngine SDK up to the front of your PYTHONPATH. That should ensure that Python finds the google.appengine package instead of the package provided by python-protobuf.

PYTHONPATH=/opt/google_appengine/google_appengine_1.2.7 \
    python dev_appserver.py helloWorld

This is a bug that should be reported to the AppEngine SDK project.

Update: I've submitted a bug against the AppEngine API.

Answer 2

It was a file permission problem. os_compat.py wasn't readable by user, just by root. I'm not sure if I screwed this up, or if the permissions by default don't have read-all, but that was the fix.

I hate to accept my own answer after Kyle gave such a good response, but I don't need the $PYTHONPATH fix to make it work now that I did sudo chown -R +r /opt/google_appengine/google_appengine_1.2.7

Answer 3

With that error, Python is saying that it can't find or read the name that it's trying to import. Since the import of os_compat is the very first executable line of AppEngine's dev_appserver.py, I suspect that there's a problem with the way that your paths are configured.

Answer 4

The latest version of Ubuntu (10.10) has also removed Python 2.5 - making it a pain to install the App Engine development environment.

I (finally) got my environment working (including using App Engine Helper for unit testing). I built this bash script which might be useful to others. It installs:

1. sqlite
2. libsqlite
3. pep8
4. mock
5. OpenSSL
6. Python 2.5.2
7. Python SSL Library
8. Django 1.1 (latest version in production)
9. App Engine
10. App Engine Helper

http://pageforest.googlecode.com/hg/tools/pfsetup

Answer 5

Ubuntu 11.04 comes with python 2.6 as the default version. It is suggested to use Google app engine with version 2.5. I am using it though for many years with python 2.6 without any issues.

What you need to do in order to execute it smoothly with python 2.6 is to edit google/appengine/tools/dev_appserver.py and add these three lines

  '_counter',
  '_fastmath',
  'strxor',

after 'XOR', and before '_Crypto_Cipher__AES', around line ~1350.

Answer 6

If you are now using Google Cloud SDK, put this into ~/.profile.

export CLOUDSDK_ROOT_DIR="/path/to/google/cloud/sdk/"
export APPENGINE_HOME="${CLOUDSDK_ROOT_DIR}/platform/appengine-java-sdk"
export GAE_SDK_ROOT="${CLOUDSDK_ROOT_DIR}/platform/google_appengine"

# The next line enables Java libraries for Google Cloud SDK
export CLASSPATH="${APPENGINE_HOME}/lib":${CLASSPATH}

# The next line enables Python libraries for Google Cloud SDK
export PYTHONPATH=${GAE_SDK_ROOT}:${PYTHONPATH}

# * OPTIONAL STEP *
# If you wish to import all Python modules, you may iterate in the directory
# tree and import each module.
#
# * WARNING *
# Some modules have two or more versions available (Ex. django), so the loop
# will import always its latest version.
for module in ${GAE_SDK_ROOT}/lib/*; do
    if [ -r ${module} ]; then
        PYTHONPATH=${module}:${PYTHONPATH}
    fi
done
unset module

Do not put inside ~/.bashrc because, every time you open a bash session, all those modules will be added again and again into your PYTHONPATH environment variable.