How do I use sklearn library and datastore memcache together?

Question

I have a BallTree model trained and persisted in pickle format.

In Google App Engine, sklearn is not supported by Python 2 standard environment. I tried Cloud ML to deploy the model but it is not working either since Cloud ML only serves predict() function. My BallTree model will call query_radius(). Hence, I moved to Python 3 standard environment where the sklearn works.

After I get the indexes from query_radius(), I will retrieve the key of the entities by the dictionary in some way. With the keys, I can now retrieve all the data from datastore.

Now, I want to utilise memcache to improve the retrieval speed of datastore. However, I found that googleappengine api is not working in Python 3 standard environment. What should I do to use the memcache? Or do I have other better options?

Answer 1

You are correct that in the Python 3 App Engine Standard Environment, memcache is not included out of the box.

The alternative is to use something like Cloud Memorystore (or really, any hosted Redis or Memcache product) and manually connect to it in your app.

For example, here's an example of how to connect to Memorystore from an App Engine Flex environment that should help get you started: https://cloud.google.com/memorystore/docs/redis/connect-redis-instance-flex