Google Cloud Platform - Datastore - Python ndb API

Question

I recently started working with the Google Cloud Platform, more precisely, with the ndb-Datastore-API. I tried to use following tutorial (https://github.com/GoogleCloudPlatform/appengine-guestbook-python.git) to get used to the API.

Unfortunately, I cannot figure out how to import the third party library tweepy into tweet.py. Google Cloud does not support tweepy so that I had to include the library manually in a folder /lib. But how do I now import the installed tweepy (pip install -t lib tweepy)?

I basically just try to put an Entity in the Google Datastore but I cannot figure out what I did wrong.

tweet.py:

    # [START imports]
from __future__ import absolute_import, print_function
import os
import urllib
from time import *
import jinja2
import webapp2
from google.appengine.api import users
from google.appengine.ext import ndb

JINJA_ENVIRONMENT = jinja2.Environment(
    loader=jinja2.FileSystemLoader(os.path.dirname(__file__)),
    extensions=['jinja2.ext.autoescape'],
    autoescape=True)
# [END imports]

# [START globalvar]
# Go to http://apps.twitter.com and create an app.
# The consumer key and secret will be generated for you after
consumer_key="KEY"
consumer_secret="SECRET"
# After the step above, you will be redirected to your app's page.
# Create an access token under the the "Your access token" section
access_token="TOKEN"
access_token_secret="SECRET"
# [END globalvar]

USERNAME = "@Seeed"

def getDate():
    # local Time
    lt = localtime()
    # get Date
    year, month, day = lt[0:3]
    date = "%02i.%02i.%04i" % (day,month,year)
    return date

# [START tweet_count_entity]
class TweetCount(ndb.Model):
    """A main model for representing an individual TweetCount entry."""
    date = ndb.DateProperty(auto_now_add=True)
    tweets = ndb.IntegerProperty(indexed=False)
    user_name = ndb.StringProperty(indexed=False)

# [END tweet_count_entity]

# [START tweet_counter]
class TweetCounter(webapp2.RequestHandler):
    """
    # Create a key for the Entity
    def tweetCount_key(date):
        date = getDate()
        return ndb.Key('TweetCount', date)"""

    # Get TweetCount for actor "user_name"
    def getTweetCount(self, user_name):
        auth = OAuthHandler(consumer_key, consumer_secret)
        auth.set_access_token(access_token, access_token_secret)
        api = tweepy.API(auth)
        user = api.get_user(user_name)
        return user.followers_count

    def get(self):
        count = self.getTweetCount(USERNAME)
        tweet_count_user_name = USERNAME
        tweet_count_tweets = count
        tweet_count = TweetCount(tweets=tweet_count_tweets, user_name=tweet_count_user_name)
        tweet_count.put()
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.write("" + USERNAME + " TweetCount: " + str(count))

# [END tweet_counter]

# [START app]
app = webapp2.WSGIApplication([
    ('/', TweetCounter),
], debug=True)
# [END app]

app.yaml:

runtime: python27
api_version: 1
threadsafe: true

# [START handlers]
handlers:
- url: /.*
  script: tweet.app
# [END handlers]

# [START libraries]
libraries:
- name: webapp2
  version: latest
- name: jinja2
  version: latest
# [END libraries]

index.yaml:

indexes:
- kind: TweetCount
  properties:
  - name: date
  - name: tweets
  - name: user_name

appengine_config.py:

from google.appengine.ext import vendor

# Add any libraries installed in the "lib" folder.
vendor.add('lib')

Thank you very much for your help. I appreciate any help and an explanation about what I did wrong.

Answer 1

Based on your tweet.py and app.yaml, I can see the 2 things that may be the reason you cannot yet use tweepy in your Python application. For the sake of being through, I'll document the complete process.

Acquire tweepy library

As Tweepy is a third party library and not a library that ships with Google App Engine, we have to package it with our Python application as you've already suggested with pip install --target lib tweepy. The directory specified after the --target option is where the third party library will be downloaded.

Vendor in the third-party directory

Now that we've downloaded the third party library, Python must be made to search this lib directory when attempting to import modules. This can be done as you've shown, using an appengine_config.py file in your application directory as indicated in Installing a library like so:

from google.appengine.ext import vendor
vendor.add('lib')

Import third-party module in the application

Python will now look in the lib directory when attempting to import modules. Thus, we add our import statement where appropriate in our application code (e.g.: tweet.py):

import tweepy

This import is not currently found in your tweet.py example.

Ensure tweepy's dependencies are also imported

Note that tweepy when imported will attempt to import the ssl module so it must be included in the libraries section of your app.yaml like so:

libraries:
- name: ssl
  version: latest

This too, was not in your example app.yaml.

With the above, one should be able to successfully deploy a python GAE application which properly imports tweepy. Note that I've not actively tested any of tweepy's features, but merely imported it successfully. Should you be seeing any errors with the above examples, I would suggested also including the following in your question:

• App Engine SDK version
• Python version
• pip version
• Stack trace of the error you are receiving when deploying or serving a HTTP response