How to handle incoming request error 'Request Too Large' in Google App Engine?

Question

We provided our GAE Servlet POST URL as Webhook to third party service that sends data. According to GAE - "Each incoming HTTP request can be no larger than 32MB."

Sometimes third party service sends data more than 40MB which gets rejected by GAE server as 'Request Too Large' error. The service retries continuously on the other end upto 100 times and blocks further requests until retries completed if webhook URL doesn't return a 200 HTTP response code.

Is it possible to handle such requests and send 200 HTTP response code with GAE?

Answer 1

You cannot circumvent the 32MB upload limit in GAE directly. But you can build a working solution with one of these options:

• Use blobstore uploads. The docs say that you cannot serve more than a 32MB chunk but you should be able to upload a larger file than that.
• let the third party services upload into cloud storage
• Let the third party service upload chunks of 30MB and reassemble the complete request after the last chunk. The Compose API endpoint should be useful for that.

Sadly all those options require the third party service to make changes in their requests. You could contact Google support and ask if they would increase the limit for your app. I have my doubts that they will do it since limiting the request size and the request duration (deadline) is a great way of making your app scaleable.