I have a Python program that sends a high volume of patch requests to the Firebase database REST API for my project, around 5-15k requests every 15 minutes. I first tried to send these simultaneously, putting each call into its own thread, and I got this error repeatedly:
Exception occurred: HTTPSConnectionPool(host='my-app.firebaseio.com', port=443): Max retries exceeded with url: /items/item-id/.json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))
When I limit the amount of items so that only ~100 requests happen each interval, there are no problems. One thing I tried was to spread out the requests so they occur over a 60 second span every 15 minutes, but that still didn't work.
The function that gets passed to each thread looks like this:
def make_request(self, val):
url = "https://my-project.firebaseio.com/items/" + self.item_id + "/.json"
try:
self.s.patch(url, json.dumps({"my-key": val}), headers=None, timeout=None)
except Exception as e:
print("Exception occurred: " + str(e))
