OBJECTIVE I have an Azure VM set up with a system assigned managed identity. I want to be able to:
Allow users to access blobs inside storage account using the VM
Ensure the users are not able to access the blobs external to the VM
Use Python - most of our users are Python but not Powershell literate.
Setup details: Storage account: sa030802util. Container: testutils. Blob: hello3.txt Managed identity and roles. VM has a system assigned managed identity and contributor, storage account contributor, storage blob data contributor roles for sa030802util.
METHODS I have tried four methods to solve this problem.
Partially successful method 1: Python. In Python, I have been able to access the sa030802util storage account using the below code, derived from link, link and link. The problem is that this uses the storage account and keys rather than relying solely on the managed identity for the VM. My fear is that this leaves the possibility that users could extract the storage keys and gain access to the blobs outside the VM.
Pros: in Python. Con: not using managed identity to authenticate. BlockBlobService can't use MSI to authenticate (yet).
Partially successful method 2: Powershell. In Powershell, I have found two ways to access the blob using the managed identity. The challenge is that neither create a credential that I can easily substitute into Python as explained below. This first method is drawn from the Microsoft-taught Pluralsight course on Implementing Managed Identities for Microsoft Azure Resources (link). It uses the Az module.
Pros: uses managed identity, relatively simple. Cons: not in Python. Does not generate a credential that could be used in Python.
Partially successful method 3: Powershell. This method is drawn from link. It uses the VM managed identity to generate a SAS credential and access Azure Storage.
Pros: uses managed identity and generates SAS credential, which is potentially valuable as BlockBlobService in Python can accept a SAS token. Cons: not in Python. Overkill for Powershell itself given method 2 above achieves the same thing with less effort. I trialled it because I wanted to see if I could extract the SAS credential for use in Python.
Unsuccessful method 4: Python and Powershell. I thought I might be able to generate a SAS token in Powershell using method 3, then slot the token in to the BlockBlobService code from method 1. What I have isn't working. I suspect the reason is that the SAS credential was created for the testutils container, and the Python BlockBlobService needs a SAS credential for the sa030802util storage account.
Pro: would allow me to rely on the managed identity of the VM to access Azure Storage. Con: doesn't work!
QUESTIONS My questions are:
Am I right in thinking it's better to rely on the VM managed identity and / or SAS credential than the account keys, if I want to make sure that users can only access the storage account inside the VM?
Is there a way to cobble together code that lets me use Python to access the data? Is method 4 promising or a waste of time?
CODE
Method 1: Python
from azure.mgmt.storage import StorageManagementClient
from azure.mgmt.storage.models import StorageAccountCreateParameters
from msrestazure.azure_active_directory import MSIAuthentication
from azure.mgmt.resource import SubscriptionClient
from azure.storage.blob import BlockBlobService
# find credentials and subscription id
credentials = MSIAuthentication()
subscription_client = SubscriptionClient(credentials)
subscription = next(subscription_client.subscriptions.list())
subscription_id = subscription.subscription_id
# find storage keys
storage_client = StorageManagementClient(credentials, subscription_id)
storage_account = storage_client.storage_accounts.get_properties("<resourcegroup>", "sa030802util")
storage_keys = storage_client.storage_accounts.list_keys("<resourcegroup>", "sa030802util")
storage_keys = {v.key_name: v.value for v in storage_keys.keys}
# create BlockBlobService and for e.g. print blobs in container
account_name = "sa030802util"
account_key = storage_keys["key1"]
container_name = "testutils"
block_blob_service = BlockBlobService(account_name = account_name, account_key = account_key)
print("List blobs in container")
generator = block_blob_service.list_blobs(container_name)
for blob in generator:
print("Blob name: " + blob.name)
The output of this code is:
List blobs in container
Blob name: hello3.txt
Method 2: Powershell
Connect-AzAccount -MSI -Subscription <subscriptionid>
$context = New-AzStorageContext -StorageAccountName sa030802util
Get-AzStorageBlob -Name testutils -Context $context
The output of this code is:
Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted
---- -------- ------ ----------- ------------ ---------- ------------ ---------
hello3.txt BlockBlob 15 application/octet-stream 2019-08-02 05:45:33Z Hot False
Method 3: Powershell
# to get an access token using the VM's identity and use it to call Azure Resource Manager
$response = Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F' -Method GET -Headers @{Metadata="true"}
$ content = $response.Content | ConvertFrom-Json
#ArmToken = $content.access_token
# to get SAS credential from Azure Resource Manager to make storage calls
## convert parameters to JSON
$params = @{canonicalizedResource="/blob/sa030802util/testutils"; signedResource="c"; signedPermission="rcwl"; signedProtocol="https"; signedExpiry="2019-08-30T00:00:00Z"}
$jsonParams = $params | ConvertTo-Json
## call storage listServiceSas endpoint to create SAS credential
$sasResponse = Invoke-WebRequest -Uri https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourceGroup>/providers/Microsoft.Storage/storageAccounts/sa030802util/listServiceSas/?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F -Method POST -Body $jsonParams -Headers @{Authorization = "Bearer $ArmToken"} -UseBasicParsing
## extract SAS credential from response
$sasContent = $sasResponse.Content | ConvertFrom-Json
$sasCred = $sasContent.serviceSasToken
# as example, list contents of container
$context = New-AzStorageContext -StorageAccountName sa030802util -SasToken $sasCred
Get-AzStorageBlob -Name testutils -Context $context
The output of this code is the same as for Method 2.
Method 4: Python and Powershell Powershell code
# to get an access token using the VM's identity and use it to call Azure Resource Manager
$response = Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F' -Method GET -Headers @{Metadata="true"}
$content = $response.Content | ConvertFrom-Json
$ArmToken = $content.access_token
# to get SAS credential from Azure Resource Manager to make storage calls
## convert parameters to JSON
$params = @{canonicalizedResource="/blob/sa030802util/testutils"; signedResource="c"; signedPermission="rcwl"; signedProtocol="https"; signedExpiry="2019-08-30T00:00:00Z"}
$jsonParams = $params | ConvertTo-Json
## call storage listServiceSas endpoint to create SAS credential
$sasResponse = Invoke-WebRequest -Uri https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourceGroup>/providers/Microsoft.Storage/storageAccounts/sa030802util/listServiceSas/?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F -Method POST -Body $jsonParams -Headers @{Authorization = "Bearer $ArmToken"} -UseBasicParsing
## extract SAS credential from response
$sasContent = $sasResponse.Content | ConvertFrom-Json
$sasCred = $sasContent.serviceSasToken
# then export the SAS credential ready to be used in Python
Python code
from azure.storage.blob import BlockBlobService, PublicAccess
import os
# import SAS credential
with open("cred.txt") as f:
line = f.readline()
# create BlockBlobService
block_blob_service = BlockBlobService(account_name = "sa030802util", sas_token=line)
# print content of testutils container
generator = block_blob_service.list_blobs("testutils")
for blob in generator:
print(blob.name)
The Python code returns the following error:
AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:<subscriptionid>
Time:2019-08-05T05:33:40.0175771Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was rcwl
2019-08-30T00:00:00.0000000Z
/blob/sa030802util/testutils
https
2018-03-28
</AuthenticationErrorDetail></Error>
