How to get filename of files stored in cloud storage and blobstore?

Question

For an application, I have upload files on both blobstore and cloud storage.

I am doing that by generating upload links by this:

upload_url = blobstore.create_upload_url('/upload', gs_bucket_name=app_identity.get_default_gcs_bucket_name())

Files are saved on both blobstore and cloud storage, and I can perform GQL queries to retrieve any files I want from blobstore. However, I am unable to find a way to get the filename in cloud storage.

When files are uploaded in cloud storage, it is assigned a long filename automatically.

I found a command, but I am not sure how to use this: https://cloud.google.com/appengine/docs/standard/python/refdocs/google.appengine.ext.blobstore.blobstore#google.appengine.ext.blobstore.blobstore.BlobInfo

When I tried:

cloud_store = []
for upload in uploads:
     gcs_link = blobstore.BlobInfo(db.get(upload)).gs_object_name
     cloud_store.append(gcs_link)

Here uploads stores result of a GQL query performed on a kind that has blobstore.BlobReferenceProperty() property.

I get this error:

BadArgumentError: Expected an instance or iterable of (<type 'basestring'>, <class 'google.appengine.api.datastore.Entity'>, <class 'google.appengine.api.datastore_types.Key'>); received <models.PhotoAlbum object at 0x00000000054CD7B8> (a PhotoAlbum).

When I tried:

cloud_store = []
for upload in uploads:
    gcs_link = blobstore.BlobInfo(upload.key()).gs_object_name
    cloud_store.append(gcs_link)

I get this error:

TypeError: Must provide Entity or BlobKey

Answer 1

Since uploads is a result of datastore query it means it contains a list of datastore entities (I assume it's not a keys_only query). So upload is one such entity, which contains the blobstore.BlobReferenceProperty() property (you didn't specify its name, let's assume it's called blob_key).

If so what you need to pass to blobstore.BlobInfo() needs to be the blob key, i.e. the upload's datastore entity property, not the key of the entity itself. So I'd try (with the above assumption about the property name):

gcs_link = blobstore.BlobInfo(upload.blob_key).gs_object_name

Answer 2

GCS via Java:

You need the Google Cloud Storage service Object which you can obtained from below a steps:

A. Configure

1. Configuration Input like googleServiceAccountCredentials and projectId

StorageOptions configureStorageOptions = 
StorageOptions.newBuilder().setProjectId(projectId).setCredentials(googleServiceAccountCredentials).build();

2. GCS service object via configureStorageOptions obtain from Step 1:

Storage getStorage = configureStorageOptions.getService();

Since from above code snippet you have Google Storage Object.

B. Use case

Here we will take an example that file uploaded to Google Storage inside bucket is based on requestNumber i.e. request number will be a directory in the Google Storage bucket.

So we must create path as shown in below code snippet:

String path = bucketName.concat("/").concat(requestNumber).concat("/");

Note:

1. Don't forget to append / as show in the above path.
2. Same path will be referred to in below code snippet.

Now you can refer the below code snippet for fetching File name from Google Storage Account:

C. Code Snippet to fetch File Name from GCS

List<String> supportingDocuments = new ArrayList<String>();
  try {
            log.info("Get FileName From Path {} With BucketName As {}", path, bucketName);
            Page<Blob> blobs = getStorage().list(bucketName, Storage.BlobListOption.currentDirectory(), Storage.BlobListOption.prefix(path));
            for (Blob blob:blobs.iterateAll()){
                String fileName = blob.getName().replace(path, "");
                supportingDocuments.add(fileName);
            }
        } catch (Exception e){
            log.error(e.getMessage());
        }

Note:

1. getStorage() method is same which is mentioned A. Configure point 2 as ref getStorage i.e. Storage Java Object