How to use Google Cloud Datastore from existing Compute Engine with different project id?

Question

Google Cloud Datastore is a great way to share information between app engine and compute engine. This instruction shows how to use google cloud datastore from compute engine, but it seems both app engine and compute engine have to be in a same project id.

My datastore inside app engine and compute engine have different project id.

Here is what i did. in googledatastore/helper.py around line 65, added 1 line

  try:
    raise client.AccessTokenRefreshError # <== added
    credentials = gce.AppAssertionCredentials(connection.SCOPE)
    http = httplib2.Http()
    credentials.authorize(http)
    credentials.refresh(http)
    logging.info('connect using compute credentials')
    return credentials
  except (client.AccessTokenRefreshError, httplib2.HttpLib2Error):
    if (os.getenv('DATASTORE_SERVICE_ACCOUNT')
        and os.getenv('DATASTORE_PRIVATE_KEY_FILE')):
      with open(os.getenv('DATASTORE_PRIVATE_KEY_FILE'), 'rb') as f:
        key = f.read()
      credentials = client.SignedJwtAssertionCredentials(
          os.getenv('DATASTORE_SERVICE_ACCOUNT'), key, connection.SCOPE)
      logging.info('connect using DatastoreSignedJwtCredentials')
      return credentials

Then ignored compute engine credential. after that just follow instruction for 'not from compute engine'. hm.. It doesn't look best way.

Is it proper way to use datastore from compute engine under another project id?

proppy proppy · Accepted Answer · 2013-10-08T18:28:20

I would recommend to consolidate Compute Engine and App Engine under the same project, but if some reason this is not possible you could try the following:

Visit the App Engine Admin Console (for your App Engine app)
Select the Application ID for which the Google Cloud Datastore API should be enabled.
Click the Application Settings link within the left hand navigation.
Scroll to the Cloud Integration section.
If no project is showing up, click Add Project. Project creation will take a few minutes, wait until you see your Google APIs Console Project Number in the Basics section of the Application Settings page.
Click on the Google APIs Console Project Number link.
Select APIs from the list of services.
Scroll to the Google Cloud Datastore API.
Switch Google Cloud Datastore API from OFF to ON.
Visit the Google Cloud Console (for your Compute project)
Select the existing Project ID for which the Google Cloud Datastore API should be enabled.
Select APIs from the list of services.
Scroll to the Google Cloud Datastore API.
Switch Google Cloud Datastore API from OFF to ON.
Go to Permissions
Copy your compute service account email address (should be something like: [email protected])
Visit the Google Cloud Console (for your App Engine app)
Go to Permissions
Add your compute service account email address

And it should hopefully work with the unmodified python client.

How to use Google Cloud Datastore from existing Compute Engine with different project id?

1 Answers