How to create directories in Azure storage container without creating extra files?

Question

I've created python code to create a range of folders and subfolders (for data lake) in an Azure storage container. The code works and is based on the documentation on Microsoft Azure. One thing though is that I'm creating a dummy 'txt' file in the folders in order to create the directory (which I can clean up later). I was wondering if there's a way to create the folders and subfolders without creating a file. I understand that the folders in Azure container storage are not hierarchical and are instead metadata and what I'm asking for may not be possible?

connection_string = config['azure_storage_connectionstring']
gen2_container_name = config['gen2_container_name']
container_client = ContainerClient.from_connection_string(connection_string, gen2_container_name)
blob_service_client = BlobServiceClient.from_connection_string(connection_string)

# blob_service_client.create_container(gen2_container_name)


def create_folder(folder, sub_folder):
    blob_client = container_client.get_blob_client('{}/{}/start_here.txt'.format(folder, sub_folder)) 

    with open ('test.txt', 'rb') as data:
        blob_client.upload_blob(data)



def create_all_folders():
    config = load_config()
    folder_list = config['folder_list']
    sub_folder_list = config['sub_folder_list']
    for folder in folder_list:
        for sub_folder in sub_folder_list:
            try:
                create_folder(folder, sub_folder)
            except Exception as e:
                print ('Looks like something went wrong here trying to create this folder structure {}/{}. Maybe the structure already exists?'.format(folder, sub_folder))

For storage sdk it is not possiable, you can use datalake sdk. — 1_1

1_1 1_1 · Accepted Answer · 2021-01-27T03:30:39

I've created python code to create a range of folders and subfolders (for data lake) in an Azure storage container. The code works and is based on the documentation on Microsoft Azure. One thing though is that I'm creating a dummy 'txt' file in the folders in order to create the directory (which I can clean up later). I was wondering if there's a way to create the folders and subfolders without creating a file. I understand that the folders in Azure container storage are not hierarchical and are instead metadata and what I'm asking for may not be possible?

No, for blob storage, this is not possible. There is no way to create so-called "folders"

But you can use data-lake SDK like this to create directory:

from azure.storage.filedatalake import DataLakeServiceClient 
connect_str = "DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net"
datalake_service_client = DataLakeServiceClient.from_connection_string(connect_str)
myfilesystem = "test"
myfolder     = "test1111111111"
myfile       = "FileName.txt"

file_system_client = datalake_service_client.get_file_system_client(myfilesystem)            
directory_client = file_system_client.create_directory(myfolder)

How to create directories in Azure storage container without creating extra files?

2 Answers