Retry Google Cloud function on arbitrary interval

Question

I've got a Google Cloud app with several cloud functions that call an API, then save the response in Firebase. I have this scheduled function set up to retry the API on error and it works great. But I want to retry the call if the data I need isn't there yet. Calling again right away could work if I throw an error, but it's highly unlikely that the missing data will be available seconds later, so I'd rather check once an hour after that until I have valid data.

Below is a shortened version of my function. I can imagine adding a setTimeout and having the function call itself again, but I'm not sure I would do that or if it's a good idea since it would keep the function alive for a long time. Is there a way to automatically retry this scheduled function on an arbitrary time interval?

exports.fetchData= functions.pubsub
    .schedule('every Tuesday 6:00')
    .timeZone('America/New_York')
    .onRun(async context => {
            const response =  fetch(...)
            .then(res => {
                if (res.status < 400) {
                    return res;
                } else {
                    throw new Error(`Network response was not ok. ${res}`);
                }
            })
            .then(res => res.json());

        const resObj = await response;

        resObj.map(x => {
            // check response for valid data
        })

        if (// data is valid) {
          // save to Firebase 
        } else {
          // retry in 1 hour
        }
    });
});

Doug Stevenson Doug Stevenson · Accepted Answer · 2019-11-02T07:18:29

Scheduled functions only run on the schedule you specify. There is no "arbitrary" scheduling. If you think that the function might frequently fail, consider just increasing the frequency of the schedule, and bail out of function invocations that don't need to run because of recent success.

If you enable retries, and the function generates an error by throwing an exception, returning a rejected promise, or timing out, then Cloud Functions will automatically retry the function on a schedule that you can't control.

setTimeout is not a feasible option to keep a function alive for longer than its configured timeout. Cloud Functions will terminate the function and all of its ongoing work after the timeout expires (and you would be paying for the time the function sits idle, which is kind of a waste).

Retry Google Cloud function on arbitrary interval

1 Answers