Google Cloud Storage - Upload Large File (4GB)

Question

I have the following script which works with small files, however fails when I try a huge file (4GB):

<?php

  require 'vendor/autoload.php';

  use Google\Cloud\Storage\StorageClient;

  $storage = new StorageClient([
      'keyFilePath' => 'keyfile.json',
      'projectId' => 'storage-123456'
  ]);

  $bucket = $storage->bucket('my-bucket');

  $options = [
      'resumable' => true,
        'chunkSize' => 200000,
        'predefinedAcl' => 'publicRead'
  ];

  // Upload a file to the bucket.
  $bucket->upload(
      fopen('data/file.imgc', 'r'),
      $options
  );

?>

The error I receive is:

Fatal error: Uncaught exception 'Google\Cloud\Core\Exception\GoogleException' with message 'Upload failed. Please use this URI to resume your upload:

Any ideas how to upload a large file?

http://googlecloudplatform.github.io/google-cloud-php/#/docs/google-cloud/v0.61.0/storage/bucket?method=upload

I've also tried the getResumableUploader():

$uploader = $bucket->getResumableUploader(fopen('data/file.imgc', 'r'), [
  'name' => 'file.imgc'
]);

try {
  $object = $uploader->upload();
} catch (GoogleException $ex) {
  $resumeUri = $uploader->getResumeUri();
  $object = $uploader->resume($resumeUri);
}

When navigating to the resume URI it returns "Method Not Allowed"

This can be done by uploading the files in chunks by this package. Example project - github.com/pionl/laravel-chunk-upload-example — Kusal Dissanayake

danemacmillan danemacmillan · Accepted Answer · 2018-05-01T23:02:18

I haven’t used this API, but I don’t think you’re supposed to just open a massive file into memory and then stuff it into this single request. You’re requesting a resumable operation, so read small portions of the file, like a couple MB at a time, and loop through it, until you have uploaded all parts of it.

Google Cloud Storage - Upload Large File (4GB)

2 Answers