Can Cloud Run respond in less than 200ms?

0
votes

Cloud Run seems to respond in over 400ms. even on repeated calls (I assume 'warmed up' containers).

I deployed a simple node.js service (see code below) that responds with a static JSON.

The service endpoint does not require user authentication. I am calling us-central1 (from Ventura County, CA. USA)

I call the service and get latency of above 400ms. Sometimes I see latency in the 5000-6000 ms. range!

I understand Cloud Run is in beta, but I'd appreciate feedback from Google on what I should expect for performance and approx. when. Also, if I am configuring something wrong, let me know. If you have better results than me - please let me know what you are doing differently.

Service I deployed: https://github.com/dorongrinstein/cloudrun-dummy-service

Test code: https://github.com/dorongrinstein/cloudrun-test

I expect the output to be in the double digit ms. range. I get an output that is in the mid 3-digit range.

FYI - I am in Ventura County, CA. My internet connection is fine. I used gcpping.com and my median latency to us-central1 is 70ms.

2
google-cloud-run
Are you accounting for the reality of the cost of cold starts in serverless environments such as Cloud Run? Accessing a function for the first time in a new server instance incurs the cost of instantiating that server instance prior to invocation. - Doug Stevenson
How do you calculate the latency? What do you see in cloud run logs? Are you interacting with other component? - guillaume blaquiere
As you can see from the code example I've provided, I do not interact with any other components. I calculate latency as is shown in my second example (see github cloudrun-test repo). - doron grinstein
See below. I submit 50 requests, then pause for 5 seconds (the container should still be up) and they submit another 50. The requests are still taking ~500ms. or more. This does not appear to be a cold/warm start issue. - doron grinstein

2 Answers

3
votes

Yes Cloud Run can respond in less than 200ms.

You seemed to be hitting a bug in our networking infrastructure, which should now be fixed.

0
votes

According to this Cloud Run engineer, you should expect similar cold start times in Cloud Run as in Cloud Functions.

One thing you could try is raising your containers' memory limit. At least in Cloud Functions startup latency has been found to depend highly on memory allocation.