Create Folders inside google cloud storage bucket using python

Question

I am trying to create a new bucket with 2 empty folders within it on Google Cloud storage using python client library.

I referred to the python client library API for GCS (https://google-cloud-python.readthedocs.io/en/latest/storage/client.html) and I found a create_bucket() method, but I would also like to create 2 folders - 'processed' and 'unprocessed' within it, but not able to find a method to create folders. Any help would be appreciated.

Thanks

Answer 1

GCS has a flat namespace, i.e., the concept of a 'folder' is not built into the service but rather an abstraction implemented by various clients. For example, both the Cloud Storage web UI (console.cloud.google.com/storage/browser) and gsutil implement the folder abstraction using an object name that ends with "/" Thus, you could create folders by creating objects like your-bucket/abc/def/ but that would only be a folder to clients that know about/support that naming convention.

Answer 2

def copyFilesInFolder(self, file_name, src_blob_name, destination_blob_name):
    """Copies a blob from one bucket to another with a new name."""
    # bucket_name = "your-bucket-name"
    # blob_name = "your-object-name"
    # destination_bucket_name = "destination-bucket-name"
    # destination_blob_name = "destination-object-name"

    # storage_client = storage.Client()

    srcBlob = src_blob_name + '/' + file_name
    destBlob = destination_blob_name + '/' + file_name
    source_blob = self.bucket.blob(srcBlob)
    destination_bucket = storage_client.bucket(destBlob)

    blob_copy = self.bucket.copy_blob(
        source_blob, self.bucket, destBlob
    )
    print(blob_copy)
    print(
        "File {} in bucket {} copied to blob {} in bucket {}.".format(
            file_name,
            src_blob_name,
            file_name,
            destination_blob_name,
        )
    )

    return True

In GCP direct folder creation concept is not there. So we can save a new file in the new folder, this way even the destination folder doesn't exist it'd be created.