Error uploading blob to storage account from celery in docker container running in azure registry

Question

Unable to upload a blob to azure from a task running on a celery worker in a docker container on Azure (from Azure registry). Able to successfully upload blob from local celery worker.

I've attempted to resolve the issue by changing the access policy in Azure to "Blob" or "Container" however this did not resolve the issue. Running task on local celery worker completes without error with all access levels. I have made sure the container exists and is spelled correctly.

Code to upload blob:

block_blob_service = BlockBlobService(account_name='my_storage_account', 
account_key= 'my_storage_key')

directory_main = demo_data['username']
filename = '\demographics.json'

block_blob_service.create_blob_from_text('container_name', directory_main + 
filename, demo_json)

When I attempt to upload through the container I get the following error message:

AzureException('The specified resource does not exist. ErrorCode: ResourceNotFound\n

The "Put" requests have two differences:

Local: 'User-Agent': 'Azure-Storage/1.4.0-1.5.0 (Python CPython 3.7.3; Windows 10)'

Container: 'User-Agent': 'Azure-Storage/1.4.0-1.5.0 (Python CPython 3.6.8; Linux 4.14.111-boot2docker)'

Also, when running locally, the "Authorization" header is generated, while when running from the container this header is missing. Any help would be greatly appreciated.

Traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/app/factory.py", line 43, in __call__
    return self.run(*args, **kwargs)
  File "/app/tasks.py", line 158, in _download_task
    raise(e)
  File "/app/tasks.py", line 107, in _download_task
    block_blob_service.create_blob_from_text('my_container', directory_main + filename, demo_json)
  File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/blockblobservice.py", line 846, in create_blob_from_text
    timeout=timeout)
  File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/blockblobservice.py", line 751, in create_blob_from_bytes
    use_byte_buffer=True
  File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/blockblobservice.py", line 582, in create_blob_from_stream
    timeout=timeout)
  File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/blockblobservice.py", line 1101, in _put_blob
    return self._perform_request(request, _parse_base_properties)
  File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 430, in _perform_request
    raise ex
  File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 358, in _perform_request
    raise ex
  File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 344, in _perform_request
    HTTPError(response.status, response.message, response.headers, response.body))
  File "/usr/local/lib/python3.6/site-packages/azure/storage/common/_error.py", line 115, in _http_error_handler
    raise ex
azure.common.AzureMissingResourceHttpError: The specified resource does not exist. ErrorCode: ResourceNotFound
<?xml version="1.0" encoding="utf-8"?><Error><Code>ResourceNotFound</Code><Message>The specified resource does not exist.
RequestId:46ba6512-301e-0061-2e62-2c37b7000000
Time:2019-06-26T21:03:00.0960002Z</Message></Error>

Peter Pan Peter Pan · Accepted Answer · 2019-06-28T06:31:19

Per my experience, there are two possible mistakes in your script.

If the container container_name does not exist in your storage account, so you need to call create_container first before create_blob_from_text. If it exists, create_container function will not make any negative consequence for your work.
```
block_blob_service.create_container('container_name')
block_blob_service.create_blob_from_text('container_name', directory_main + 
```
filename, demo_json)
Please refer to the offical document Naming and Referencing Containers, Blobs, and Metadata, if you want to set some virtual directories for your blob, the correct way is to use / not \. So as the code below.
```
directory_main = demo_data['username']
filename = '/demographics.json' # not '\demographics.json'
```

Hope it helps.

Error uploading blob to storage account from celery in docker container running in azure registry

1 Answers