I have a Firebase function that is triggered by a PubSub message and processes an array of calls that need to be made to the Google Drive API. When calls to the Google Drive API fail due to rate limit errors I want to retry these calls. I add the calls to a new PubSub message and add this to the queue. This message will be delivered immediately. Therefore I set a delay until parameter in the message and only process the message if the delay has passed and otherwise add the message to the queue again. This result in the message being delivered and retried many times. Is there a way in Firebase functions to have the listener not accept the message? I can then set the retry delay on the PubSub queue.
1 Answers
0
votes
I would look at this from either the Google Drive API side or the PubSub side.
First suggestion is to use exponential backoff when calling the Google Drive API when getting rate limited, as it says in the API docs.
The other suggestion is to only ACK the messages from PubSub after your call to the Google Drive API was successful. Depending on your acknowledgement deadline settings this will basically block the message from reappearing in the queue until you ACK or until the deadline runs out. If you don't ACK the message, it will get tossed back into the queue automagically after the deadline passes.
Situations like yours are the reason this ACK mechanism is in place.
