Error powershell during HDInsight command with cert for Datalakestore connection

Question

Am trying to create an HDInsight cluster using powershell with a datalakestore as an additional storage. Am able to create the cluster using portal by uplaoding the cert in pfx format. Powershell command give me error, screenshot attached.

New-AzureRmHDInsightClusterConfig `
        | Add-AzureRmHDInsightClusterIdentity `
            -ObjectID $objectId `
            -AadTenantId $tenantId `
            -CertificateFilePath $certificateFilePath `
            -CertificatePassword $certificatePassword `
        | New-AzureRmHDInsightCluster `
            -ClusterName $clusterName `
            -ResourceGroupName $clusterResourceGroupName `
            -HttpCredential $httpCredentials `
            -SshCredential $sshCredentials `
            -Location $location `
            -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
            -DefaultStorageAccountKey $storageAccountKey `
            -DefaultStorageContainer $storageAccountContainer `
            -ClusterSizeInNodes $clusterNodes `
            -ClusterType Spark `
            -Version "3.6" `
            -OSType Linux

I also tried the below.

New-AzureRmHDInsightCluster `
-ClusterName $clusterName `
-ResourceGroupName $clusterResourceGroupName `
-HttpCredential $httpCredentials `
-SshCredential $sshCredentials `
-Location $location `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKey `
-DefaultStorageContainer $storageAccountContainer `
-ClusterSizeInNodes $clusterNodes `
-ClusterType Spark `
-Version "3.6" `
-OSType Linux `
-ObjectID $objectId `
-AadTenantId $tenantId `
-CertificateFilePath $certificateFilePath `
-CertificatePassword $certificatePassword

I see an error below. Not sure what am missing. the error does not give me much insight(no pun intended ;)). screenshot

I can create the cluster without the certificate though. I also want to be able to give -CertificateFileContents property and pass the byte[] read certificate retrieved from azure key vault, once am able to get this working. Any pointers would be great.

Vijay Veera Vijay Veera · Accepted Answer · 2018-01-16T02:20:37

Am able to resolve this error. The objectId I provided was application ObjectId, instead when I provided the objectId of Service Principal corresponding to the ADApp, it worked.

But when i try to provide the parameter -CertificateFileContents and input the byte[] retrieved from keyvault, the cluster is created but the Service Principal setting to access DLS didn't get configured.Not sure what setting is going wrong here. Am converting the cert content to byte array before passing as parameter.

$cert = Get-AzureKeyVaultSecret -VaultName 'keyvault' -Name $certName
$certBytes = [System.Convert]::FromBase64String($cert.SecretValueText)

New-AzureRmHDInsightCluster `
    -ClusterName $clusterName `
    -ResourceGroupName $clusterResourceGroupName `
    -HttpCredential $httpCredentials `
    -SshCredential $sshCredentials `
    -Location $location `
    -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
    -DefaultStorageAccountKey $storageAccountKey `
    -DefaultStorageContainer $storageAccountContainer `
    -ClusterSizeInNodes $clusterNodes `
    -ClusterType Spark `
    -Version "3.6" `
    -OSType Linux `
    -ObjectID $objectId `
    -AadTenantId $tenantId `
    -CertificateFileContents $certBytes

Error powershell during HDInsight command with cert for Datalakestore connection

1 Answers