Google Drive resumable upload with javascript

Question

I'm trying to upload files to Google Drive using Google APIs Client Library for JavaScript and resumable upload type.

I authenticate and get the upload URI successfully, but I ran into problems while sending the actual data. If the file contains only ASCII characters, the file is sent successfully to Drive, but in case of special characters (åäö) or binary file (such as PNG) the file gets corrupted. My guess would be that somewhere in the process the file is encoded to unicode in client side.

If I use "btoa()" to encode the raw data to base64 and add header "Content-Encoding: base64" to the data sending request, the file uploads fine. Using this method however increases the overhead for 33%, which is quite a lot when the planned upload size of files is 100MB to 1GB.

Here are some code examples:

Getting the resumable upload URI:

// Authentication is already done
var request = gapi.client.request({
    "path": DRIVE_API_PATH, // "/upload/drive/v2/files"
    "method": "POST",
    "params": {
        "uploadType": "resumable"
    },
    "headers": {
        "X-Upload-Content-Type": self.file.type,
        //"X-Upload-Content-Length": self.file.size
        // If this is uncommented, the upload fails because the file size is
        // different (corrupted file). Manually setting to the corrupted file
        // size doesn't give 400 Bad Request.
    },
    "body": {
        // self.file is the file object from <input type="file">
        "title": self.file.name, 
        "mimeType": self.file.type,
        "Content-Lenght": self.file.size,
    }
});

Sending the whole file in one go:

// I read the file using FileReader and readAsBinaryString
// body is the reader.result (or btoa(reader.result))
// and this code is ran after the file has been read
var request = gapi.client.request({
    "path": self.resumableUrl, // URI got from previous request
    "method": "PUT",
    "headers": {
        //"Content-Encoding": "base64", // Uploading with base64 works
        "Content-Type": self.file.type
    },
    "body": body
});

Am I missing something? Is it possible to upload file in binary stream? I am new to uploading files in HTML and Javascript and I haven't found any examples using Google Javascript library with resumable upload. There is similar question in SO with no answers.

Answer 1

Blob types are a hot topic for XMLHttpRequest implementations and they are not truly mature. I'd recommend you to stick with base64 encoding. Google's JavaScript client lib doesn't support resumable uploads because it's very unlikely that a client side browser app uploads very large files directly to Google Drive.

Answer 2

What works

To upload a binary blob, use github/googleapi's cors-upload-sample or use my gist fork, UploaderForGoogleDrive, which will grab access_token out of the gapi client for you.

Here is an ugly mixture of Promise and callback code that works for me. As a prerequisite, gapi,UploaderForGoogleDrive, JSZip need to be loaded via <script> tags. The snippet also omits gapi initialization and the API secrets, which are also necessary.

function bigCSV(){  // makes a string for a 300k row CSV file
    const rows = new Array(300*1000).fill('').map((v,j)=>{
      return [j,2*j,j*j,Math.random(),Math.random()].join(',');
    });
    return rows.join("\n");
}

function bigZip(){  // makes a ZIP file blob, about 8MB
    const zip = new window.JSZip();
    zip.folder("A").file("big.csv", bigCSV());
    return zip.generateAsync({type:"blob", compression:"DEFLATE"});
    // returns Promise<blob>
}

function upload2(zipcontent){
   'use strict';
    const parent = 'root';
    const spaces = 'drive';
    const metadata = {
      name: 'testUpload2H.zip',
      mimeType: 'application/zip',
      parents: [parent]
    };
    const uploader = new window.UploaderForGoogleDrive({
      file: zipcontent,
      metadata: metadata,
      params: {
        spaces,
        fields: 'id,name,mimeType,md5Checksum,size'
      },
      onProgress: function(x){
         console.log("upload progress:",Math.floor(100*x.loaded/x.total));
      },
      onComplete: function(x){
        if (typeof(x)==='string') x = JSON.parse(x);
        // do something with the file metadata in x
        console.log("upload complete: ");
      },
      onError: function(e){ console.log("upload error: ",e); }
    });
    uploader.upload();
}

function uploadZipFile(){
    'use strict';
    (bigZip()
      .then(upload2)
    );
}

What doesn't work

As of Nov 2017, uploading a binary blob with the gapi.client.request call is not going to work, because of an issue where gapi removes the PUT payload

I've also tried using base64 with gapi, which works. but deposits base64 files, not true binaries; and the fetch API in cors mode, which half-worked but produced CORS-related errors and response hiding, at least for me.