Google Drive API cannot handle large App Script project upload

Question

I could not find any documentation on limits for App Script project file size, but it seems that is my problem. I have a large JS file that must be added to a App Script project - the resulting JSON for the project has 5.4 MB. I tried to use the web editor to add the new JS file, but since it is too large the browser cannot handle it. Then I though of exporting the App Script file using Google API, adding the new content and then importing the result to Drive again, using another API, as described here: https://developers.google.com/apps-script/guides/import-export

I tried to upload using uploadType=media, and it fails with a 408 (Request Timeout) error. I assumed that is because the file is larger than 5MB, since the documentation recommends using uploadType=resumable. Then I tried the latter type following both approaches: a single request - which fails again, as expected because of the file size, and as multiple requests, which in theory should work. But I may attempts timed out after uploading the initial 5MB, returning 408 after a few minutes.

It seems the API is able to upload the initial 5MB, but get lost after that. All I read about Drive makes me believe that is not a Drive limitation, so I suspect that is a App Script limitation.

Here are the steps:

Create the resumable upload session

curl -X PUT \
-H "Authorization: $GOOGLE_BEARER_TOKEN" \
-H "X-Upload-Content-Type: application/vnd.google-apps.script+json" \
-H "X-Upload-Content-Length: 5415490" \
-H "Content-Type: application/json; charset=UTF-8" \
-H "Content-Length: 23" \
-d "{\"title\": \"MyProject\"}" \
-v "https://www.googleapis.com/upload/drive/v2/files/###FILE_ID###?uploadType=resumable"

Receive the upload ID and URL

HTTP/2 200 
x-guploader-uploadid: ###UPLOAD_ID###
location: https://www.googleapis.com/upload/drive/v2/files/###FILE_ID###?uploadType=resumable&upload_id=###UPLOAD_ID###

Store the URL

GOOGLE_UPLOAD_LOCATION=https://www.googleapis.com/upload/drive/v2/files/###FILE_ID###?uploadType=resumable&upload_id=###UPLOAD_ID###

Send the first chunk

curl -X PUT \
-H "Authorization: $GOOGLE_BEARER_TOKEN" \
-H "Content-Type: application/vnd.google-apps.script+json" \
-H "Content-Length: 5242879" \
-H "Content-Range: bytes 0-5242878/5415490" \
-T file_chunk_1 \
-v "$GOOGLE_UPLOAD_LOCATION"

Success: the first 4980736 bytes were received

HTTP/2 308 
range: bytes=0-4980735

Send the first chunk, with the remaining bytes

curl -X PUT \
-H "Authorization: $GOOGLE_BEARER_TOKEN" \
-H "Content-Type: application/vnd.google-apps.script+json" \
-H "Content-Length: 434754" \
-H "Content-Range: bytes 4980736-5415489/5415490" \
-T file_chunk_2 \
-v "$GOOGLE_UPLOAD_LOCATION"

After some minutes a timeout

HTTP/2 408

Checking the upload status:

curl -X PUT \                          
-H "Authorization: $GOOGLE_BEARER_TOKEN" \
-H "Content-Length: 0" \
-H "Content-Range: bytes */5415490" \
-v "$GOOGLE_UPLOAD_LOCATION"

I see that only the first 5MB were uploaded:

HTTP/2 308
range: bytes=0-5242879

And all following attempts return the same range, after the timeout.

You should rather create a issue in issuetracker. See tag info page for more details. Wondering what you have in a 5mb js file though? — TheMaster
Not the best design, but it is a list of data that the script uses. I am already considering to place it somewhere else. — MarceloCosta

MarceloCosta MarceloCosta · Accepted Answer · 2020-04-23T07:53:42

Apparently the problem was the file I was uploading itself. I realized I was uploading one file inside the project (i.e. inside the project JSON file) against an ID that I have deleted in one of my several tests. The process is so error prone that I did not see that.

After fixing that and halting an original chunk, I could check the pending bytes and successfully send the second chunk completing the upload.

Google Drive API cannot handle large App Script project upload

1 Answers