Stream Files to Zip File in Azure Blob Storage using Python?

Question

I have the following problem in Python:

I am looking to create a zipfile in Blob Storage consisting of files from an array of URLs but I don't want to create the entire zipfile in memory and then upload it. I ideally want to stream the files to the zipfile in blob storage. I found this write up for C# https://andrewstevens.dev/posts/stream-files-to-zip-file-in-azure-blob-storage/ as well as this answer also in C# https://stackoverflow.com/a/54767264/10550055 .

I haven't been able to find equivalent functionality in the python azure blob SDK and python zipfile library.

Stanley Gong Stanley Gong · Accepted Answer · 2021-05-22T07:15:27

Try this :

from zipfile import ZipFile
from azure.storage.blob import BlobServiceClient
import os,requests


tempPath = '<temp path>'

if not os.path.isdir(tempPath):
    os.mkdir(tempPath)

zipFileName = 'test.zip'

storageConnstr = ''
container = ''

blob = BlobServiceClient.from_connection_string(storageConnstr).get_container_client(container).get_blob_client(zipFileName)


fileURLs = {'https://cdn.pixabay.com/photo/2015/04/23/22/00/tree-736885__480.jpg',
'http://1812.img.pp.sohu.com.cn/images/blog/2009/11/18/18/8/125b6560a6ag214.jpg',
'http://513.img.pp.sohu.com.cn/images/blog/2009/11/18/18/27/125b6541abcg215.jpg'}



def download_url(url, save_path, chunk_size=128):
    r = requests.get(url, stream=True)
    with open(save_path, 'wb') as fd:
        for chunk in r.iter_content(chunk_size=chunk_size):
            fd.write(chunk)

zipObj = ZipFile(tempPath + zipFileName, 'w')

#download file and write to zip
for url in fileURLs:
    localFilePath = tempPath + os.path.basename(url)
    download_url(url,localFilePath)
    zipObj.write(localFilePath)
    
zipObj.close()

#upload zip
with open(tempPath + zipFileName, 'rb') as stream:
    blob.upload_blob(stream)

Stream Files to Zip File in Azure Blob Storage using Python?

1 Answers