Upload a file to Google Cloud Storage (Java)

Question

Basically, the issue is in the title: I cannot figure out how to upload a file to Google Storage

I have checked this webpage:
https://cloud.google.com/storage/docs/uploading-objects

These two lines are suggested for "uploading objects" in Java language tab:

InputStream content = new ByteArrayInputStream("Hello, World!".getBytes(UTF_8));
Blob blob = bucket.create(blobName, content, "text/plain");

The bucket object above can be received from storage object, and that's actually the root of the problem

In all tutorials I have seen so far (particularly in this one: https://support.google.com/dfp_premium/answer/1733127?hl=en), the storage belongs to com.google.api.services.storage.Storage instead of com.google.cloud.storage.Storage, and is instantiated as follows:

        httpTransport = GoogleNetHttpTransport.newTrustedTransport();

        GoogleCredential credential = new GoogleCredential.Builder()
            .setTransport(httpTransport)
            .setJsonFactory(JacksonFactory.getDefaultInstance())
            .setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
            .setServiceAccountScopes(
                Collections.singleton(STORAGE_SCOPE))
            .setServiceAccountPrivateKeyFromP12File(p12File).build();

        storage = new com.google.api.services.storage.Storage.Builder(httpTransport,
            JacksonFactory.getDefaultInstance(), credential)
            .setApplicationName(PROJECT_NAME).build();

After creating a storage object in this way, we may obtain a bucket object as follows:

    com.google.api.services.storage.Storage.Buckets.Get object = storage.buckets().get(BUCKET_NAME);        
    com.google.api.services.storage.model.Bucket bucket = object.execute();

But I can't use com.google.api.services.storage.model.Bucket for uploading files because it doesn't have create() method mentioned above. I need to use com.google.cloud.storage.Bucket instead.

I have checked this stackoverflow question too: Upload image to Google Cloud Storage (Java)
and the examples in the provided links indeed use com.google.cloud.storage.Storage and com.google.cloud.storage.Bucket but they never show HOW to pass credentials and instantiate storage object similar to the code above.

Could someone provide a clear and comprehensive Java snippet showing how to upload a file to Google Cloud Storage, including all necessary credential instantiations of storage object?

EDIT. In other words, I need to customize the instantiation of com.google.cloud.storage.Storage like above, instead of using this line: Storage storage = StorageOptions.getDefaultInstance().getService();

EDIT2. Finally I was able to upload the file after retrieving the bucket by the following code now:

GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(KEY_JSN)).createScoped(Lists.newArrayList("https://www.googleapis.com/auth/devstorage.read_write"));

Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

Bucket bucket = storage.get(BUCKET_NAME);

One single subtle point remaining for me, is that the following line:

Page<Bucket> buckets = storage.list();

returns an empty list. Does it make sense to receive an empty bucket list while being able to receive a particular bucket meta by get(bucket_name) ?

Answer 1

I was able to create and upload to a bucket using the com.google.cloud.storage libraries in the following snippet derived from the documentation:

public void createBlobFromByteArray(String blobName, String jsonPath, String bucketName) throws IOException {
  GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(jsonPath)).createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));

  Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();

  Bucket bucket = storage.create(BucketInfo.of(bucketName));

  Blob blob = bucket.create(blobName, "Hello, World!".getBytes(UTF_8), "text/plain");
}

You need to create a JSON service account key in and download it from the Google Cloud console. Then pass it in the GoogleCredentials method to authenticate 'storage'

Answer 2

Did you ever figure out the cause ? storage.list() returns the list of buckets in the project. It works fine for me.

EDIT#1: Did you trying using Google API Client libraries instead of the Cloud Client libraries ?

Example code below:

     try {
        httpTransport = GoogleNetHttpTransport.newTrustedTransport();

          // Authenticate using Google Application Default Credentials.
          GoogleCredential gCredential = GoogleCredential.getApplicationDefault();
          if (gCredential.createScopedRequired()) {
            List<String> scopes = new ArrayList<>();
            // Set Google Cloud Storage scope to Full Control.
            scopes.add(ComputeScopes.DEVSTORAGE_FULL_CONTROL);
          }         

          // Create Storage object for listing buckets.
          Storage storage =
              new Storage.Builder(httpTransport, JSON_FACTORY, gCredential)
                  .setApplicationName(APPLICATION_NAME)
                  .build();     

        Buckets buckets = storage.buckets().list(projectId).execute(); <-- give your project identifier.
        for(Bucket buckt:buckets.getItems()) {
            System.out.println(buckt.getName());
        }

    } catch (GeneralSecurityException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

If you haven't tried you can give this a try and see if this works. If not, can be something with your buckets. If works, then most possibly we may have a bug in the cloud client libs.

Answer 3

• Export the Google credential JSON from command line:

export GOOGLE_APPLICATION_CREDENTIALS='\path\key.json'

• Run this code in application:

Storage storage = StorageOptions.getDefaultInstance().getService();
BlobId blobId = BlobId.of(BUCKET_NAME, OBJECT_NAME);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "Hello, World!".getBytes(UTF_8));