GAE Performance with Datastore queries

Question

I am experiencing decreased performance when my application is deployed on GAE. For example, a GET request that takes around 190ms on the localhost, it takes 2000ms when deployed on GAE.

As it can be observed from the Traces report, the query itself is taking only 30ms, but the time spent on the network is high. There were no concurrent requests and my instance is F1 type. I tried it multiple times and 2000ms seems to be the average time for this request. Any ideas what I should look into?

in general one shouldn't compare performance on the local server with that on GAE - GAE performance depends on the instance class and is usually lower than even moderately powerful laptops these days. Plus local network propagation delays are typically neglijible compared to GAE — Dan Cornilescu
Is your instance running when you make this call? The logs will show a message saying something like "This request caused a new instance to be started...". — Andrei Volgin
@DanCornilescu yes, that's true. But in this case the difference is too high and waiting 2s for such small data is too much — Roxana Roman
Is this behaviour seen on other endpoints also? If yes, maybe it's the time waiting for a "slot" to be available in the instance i.e having too many concurrent requests, without fulfilling the conditions to start a new. — nizz

Eric Simonton Eric Simonton · Accepted Answer · 2016-08-20T13:30:14

The trace will not record "time on the network" the way I think you're thinking. It starts when the request reaches your server, and ends when your server finishes processing it. Network lag between client and server is not recorded.

Since the first RPC is about 1.9 seconds into the request, it shows that all the time is being spend in your app itself. Not waiting for network IO to the datastore or memcache, but just your server running code locally. My first thought was the same as Andrei in the comments: it looks like a trace from the first request to the instance, where it can spend a lot of time initializing your server. Please double-check to be sure that is not the case.

After that, my next step would be to narrow down exactly what part of the code is taking so long. You could do that with logging at different points in your request handling code. You might also be able to sleuth out some hints using the Stackdriver Debugger, which is a very slick tool.

GAE Performance with Datastore queries

1 Answers