Local development with Cloud Tasks & Cloud Datastore with GAE with Python3

6
votes

Context:- We are using GAE with Python3 and so GAE APIs package isn't available so we are using google-cloud-* packages for interacting with GAE services

i.e. google-cloud-tasks for push queues, google-cloud-datastore for datastore.

Problem:- There is no way to test things in development environment as google-cloud-* packages directly act on production services.
i.e. if I push a task using google-cloud-tasks it would push in production queue, similarly if I create or update an entity from development environment it would be updating entity in production datastore.

Earlier with GAE APIs packages in local system it used to have local cloud tasks and datastore for development purpose.

I see it as a big and very common issue, I wonder if someone else as well faced such issue and found any solution to this.

3
google-app-enginegoogle-cloud-platformgoogle-cloud-datastore
As someone who've been using python2.7 App Engine for ~10+ years, I'm perplexed by this approach too - no more local development it seems ... – Kaan Soral
Yeah, checked with google cloud support guys about it and last time I checked this wasn't in their scope of things. – vinit payal

3 Answers

5
votes

For Cloud Datastore you can follow the instructions at https://cloud.google.com/datastore/docs/tools/datastore-emulator to use the local emulator instead of your production Datastore database.

As noted in https://cloud.google.com/tasks/docs/migrating, Cloud Tasks is not currently supported in an emulator.

1
votes

I built an in-process emulator for Python development.

See also some emulators that run in a separate process in localhost: Potato London’s gcloud-tasks-emulator, mentioned in the answer above, and Aert van de Hulsbeek’s cloud-tasks-emulator.

0
votes

This Local Emulator for Google Cloud Tasks worked for me.

pip install gcloud-tasks-emulator
gcloud-tasks-emulator start --port=9090

// Note -: by default the gcloud-tasks-emulator command does not available globally switch installation directory.

/Users/{userName}/Library/Python/3.7/bin
./gcloud-tasks-emulator start --port=9090

Now we can add code changes to support cloud task in local env.

import grpc
from google.cloud.tasks_v2 import CloudTasksClient
from google.cloud.tasks_v2.gapic.transports.cloud_tasks_grpc_transport import C 
loudTasksGrpcTransport

client = CloudTasksClient(
 transport=CloudTasksGrpcTransport
 (channel=grpc.insecure_channel("127.0.0.1:9090"))
)

visit this link for complete instruction https://pypi.org/project/gcloud-tasks-emulator/