GCP Cloud Run 503 Error: Service Unavailable for long running request

Question

I have 4 API endpoints build in NodeJS, and deployed in GCP Cloud using docker image of the application which is as below:

My application, hits separate APIs, returns response and saves data in Cloud SQL.

FROM node:12

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install --only=production

# Copy local code to the container image.
COPY . ./

# Run the web service on container startup.
CMD [ "node", "app.js" ]

1 API takes around 1 min to return response and I get successful response back when trying from Postman and hitting the Cloud run URL

other 3 APIs takes 2 mins and other 20mins each to return response back and till that time I get error from Cloud run

503 Service Unavailable 
takes: 2 m 1.02 s 
478 B

Below are the configuration given :

Memory allocated: 8 GB
CPU allocated: 4 
Maximum request per container: 20
Request timeout: 3600 seconds

NOTE: I am only hitting one request at a time

What can be the problem ? I increased concurrency but that didn't work. For my application in Cloud Function service won't work as timeouts is 9 mins.

Which service I can use in GCP for this kind of work?

The 503 sounds like a crash. Are you sure that you don't reach the max memory? Do you have other log trace in Cloud Logging? Do you have other app timeouts (when you establish the database connexion for example, and a timeout could trigger a not caught exception, and thus a crash)? — guillaume blaquiere

techGaurdian techGaurdian · Accepted Answer · 2021-01-20T21:53:31

I found that the problem was with NodeJS request timeout which was 120 seconds default and with help of below link I got my issue fixed.

https://github.com/GoogleCloudPlatform/esp-v2/issues/438

So I have to do the following in my code:

I can set the timeout either for entire server:

var server = app.listen();
server.setTimeout(500000); // In milliseconds

or just for specific route:

app.post('/xxx', function (req, res) {
   req.setTimeout(500000);
});

This resolved the issue.

GCP Cloud Run 503 Error: Service Unavailable for long running request

1 Answers