How to organize libs with Python + Google App Engine + oauthlib

Question

My problem is following error:

ImportError: No module named oauthlib.oauth2

So as it appears I can't import oauthlib in gae dev environemnt

I have following project structure

app_root/
    app/          - source files of the app
    libs/         - third party libraries
      gdata/      - google data library
      atom/       - requires by google data
      oauthlib/   - [oauthlib][1]
    test/         - unit tests
app.yaml          - gae config
main.py           - main script

Here is the source of the app.yaml

application: app
version: v1

runtime: python27
api_version: 1
threadsafe: yes

handlers:
- url: /favicon\.ico
  static_files: favicon.ico
  upload: favicon\.ico

- url: .*
  script: main.app

libraries:
- name: webapp2
  version: "2.5.2"

In main.py I have following code to pick up on 'libs' folder:

sys.path.append(os.path.join(os.path.dirname(__file__), 'libs'))

and finally I have following import in my module:

from oauthlib.oauth2 import RequestValidator, WebApplicationServer

as result when I start application from pycharm I get this error:

    from oauthlib.oauth2 import RequestValidator, WebApplicationServer
ImportError: No module named oauthlib.oauth2

I really do not understand why it happens, in my IDE it doesn't highlight that import as red, so it can see that reference is good.

I am kinda new to python and all this approach with third-party libraries really confuses me. In fact there a lot of questions about this, like is there a better way to manage third-party libraries for gae.

But lets focus on the main issue. Why it throws ImportError when reference should be fine?

UPDATE #1 If I make libs as a python package, so there init.py It breaks imports inside library code, i.e. file oauthlib/oauth2/rfc6749/clients/base.py contains:

from oauthlib.oauth2.rfc6749 import tokens
from oauthlib.oauth2.rfc6749.parameters import prepare_token_request
from oauthlib.oauth2.rfc6749.errors import TokenExpiredError
from oauthlib.oauth2.rfc6749.errors import InsecureTransportError
from oauthlib.oauth2.rfc6749.utils import is_secure_transport

But current module path includes libs.prefix

mgilson mgilson · Accepted Answer · 2013-12-11T00:43:58

IIRC, you should be able to do this with a simple __init__.py placed in the right place in the path:

app_root/
    app/          - source files of the app
    libs/         - third party libraries
      __init__.py - # Add this
      gdata/      - google data library
      atom/       - requires by google data
      oauthlib/   - [oauthlib][1]
    test/         - unit tests
app.yaml          - gae config
main.py           - main script

Now, rather than messing with sys.path, you can do something like:

from libs.oauthlib.oauth2 import RequestValidator, WebApplicationServer

If you really want to keep playing with your system's path, then I think you need another os.path.dirname in there since your .py files seem to be in app. As it is, you seem to be looking for lib in the same directory as the python source files.:

app_dir = os.path.dirname(__file__)
root_dir = os.path.dirname(app_dir)
sys.path.append(os.path.join(root_dir, 'libs'))

How to organize libs with Python + Google App Engine + oauthlib

1 Answers