ARM Deployment using KeyVault and certificates for Azure Data Lake Store

Question

I basically want to create my HDI/Spark Cluster which accesses an Azure Data Lake Store by using ARM templates and also Azure Key Vault.

So far I created the cluster manually and stored the ARM template. Then I tried to populate the sensitive values from Azure Key Vault but I am struggeling how to pass in the "identityCertificate" correctly.

I also followed this steps to create the Certificate and everything: https://github.com/Azure/azure-quickstart-templates/tree/master/201-hdinsight-datalake-store-azure-storage and then this steps to upload the certificate into the KeyVault: https://blogs.technet.microsoft.com/kv/2016/09/26/get-started-with-azure-key-vault-certificates/

However, referenceing the KeyVault secret in my ARM template always ends up in this error:

{ "status": "Failed", "error": { "code": "ResourceDeploymentFailure", "message": "The resource operation completed with terminal provisioning state 'Failed'.", "details": [ { "code": "InvalidDocumentErrorCode", "message": "DeploymentDocument 'AmbariConfiguration_1_7' failed the validation. Error: 'Error while getting access to the datalake storage account gbhdi: The specified network password is not correct.\r\n.'" } ] } }

doing everything manually in the Azure Portal using same certificate etc. works just fine I also tried to set the "identityCertificate" parameter manually by using the Base64 encoded value of my certificate but this did not work either

Which value would I need to pass to my parameter if I hard-code it?

Gerhard Brueckl Gerhard Brueckl · Accepted Answer · 2017-08-18T11:57:29

seems like I found the issue and it is actually related to the previously failed ARM deployments which leave some fragments of the HDI cluster and new deployments do not overwrite these fragments but use the old settings

after deleting the cluster (which was not working anyway) I could deploy it as expected.

However, it is worth mentioning that the certificate has to be stored in KeyVault as Secret and not as Key and that it has to be base64 encoded!

here is the PowerShell script that I used:

#Add Certificate to KeyVault

$base64Cert = [System.Convert]::ToBase64String((Get-Content $certFilePath -Encoding Byte))
$base64Cert | Out-File $certFilePath.Replace(".pfx", ".base64.txt")
$cer3 = Set-AzureKeyVaultSecret -VaultName $vaultName -Name $certName -

SecretValue (ConvertTo-SecureString –String $base64Cert –AsPlainText –Force)

hope that helps other people facing the same issue!

-gerhard

ARM Deployment using KeyVault and certificates for Azure Data Lake Store

2 Answers