Google Cloud Platform: Cloud Functions vs App Engine

0
votes

This may be the wrong place for this question, so please re-direct me if necessary.

I have deployed a couple simple functions using Google Cloud Functions that do the following:

  1. Read files from AWS and write to Cloud SQL
  2. Aggregate Cloud SQL data and write csv file to Cloud Storage bucket
  3. Simple OLS prediction model on aggregated data

I have these as separate functions because (1) often takes longer than the Cloud Function maximum timeout. Because of this, I am considering moving this whole thing to App Engine as a service. My question about App Engine Standard are:

  1. What do the request timeouts mean? If I were to run this service, do I still have a short time-limit after which it will no longer run?
  2. Is App Engine the best thing to use for this task?

Thanks for all your help

2
google-app-enginegoogle-cloud-platformgoogle-cloud-functions
Google Cloud Run is the big brother to Cloud Functions and supports a timeout of 60 minutes. cloud.google.com/run/docs/configuring/request-timeout – John Hanley
If you would like to use cloud functions, here are a few SO answers, where a few cloud functions are used coherently to achieve a goal, that cannot be achieved by one cloud function due to various limitations: stackoverflow.com/questions/66050709/… and stackoverflow.com/questions/66434862/… – al-dann

2 Answers

1
votes

According to Google Documentation, GAE Standard has a maximum timeout of 1 minute for http requests and 10 minutes for cron/tasks for the older environments. Newer env have it as 10 minutes for both http requests & tasks. If your functions are taking longer than these, then GAE standard won't work for you. For such a case, you should take a look at GAE Flex - see this Google documentation which compares Flex to Standard).

Secondly, it seems to me that what you have are endpoints that are only hit occasionally or at specific scheduled times. If that is the case, I would also recommend taking a look at Cloud Run. We have a blog article about it and we have this

....Another thing to note about Cloud Run is that it only runs when it receives an HTTP request. It plays dead and comes alive to execute your code when an HTTP request comes in. When it is done executing the request, it goes 'dead' again till the next request comes in. This means you're not paying for time spent idling i.e. when it is not doing anything....

1
votes
  1. You can keep your Cloud Functions and the strong cohesion implemented by each of your 3 Functions, then you can use Cloud Workflows a serverless solution to orchestrate the 3 CF calls. The drawback : you pay for 3 CF invocations and 3 Workflows steps. But does it matter ? Since 2millions CF invocations are free and 5000 Workflows steps are free.
  2. As proposed by @NoCommandLine Cloud Run is indeed an alternative, with its timeout of 3600s(1h). The drawback: you need to wrap your code in a http request and provide a webserver like express or gunicorn.
  3. A hack is to build a docker container for your code with no need for a webserver and run it using Cloud Build which have a timeout of 24 hours.