Azure devops terraform pipeline generate client id and secret

Question

I am using this terraform manifest to deploy AKS on Azure. I can do this via the commandline fine and it works, as I have azure cli configured on my machine to generate client id and secret

https://github.com/anubhavmishra/terraform-azurerm-aks

However, I am now building this on Azure Devops Pipeline

So, far i have managed to run terraform init and plan with backend storage on Azure, using Azure Devops using this extension

https://marketplace.visualstudio.com/items?itemName=charleszipp.azure-pipelines-tasks-terraform

Question: How do i get client id and secret on the Azure devops pipeline and set that as an environment variable for terraform? I tried creating a bash az command in the pipeline

> az ad sp create-for-rbac --role="Contributor"
> --scopes="/subscriptions/YOUR_SUBSCRIPTION_ID"

but failed with this error

> 2019-03-27T10:41:58.1042923Z 
2019-03-27T10:41:58.1055624Z Setting AZURE_CONFIG_DIR env variable to: /home/vsts/work/_temp/.azclitask
2019-03-27T10:41:58.1060006Z Setting active cloud to: AzureCloud
2019-03-27T10:41:58.1069887Z [command]/usr/bin/az cloud set -n AzureCloud
2019-03-27T10:41:58.9004429Z [command]/usr/bin/az login --service-principal -u *** -p *** --tenant ***
2019-03-27T10:42:00.0695154Z [
2019-03-27T10:42:00.0696915Z   {
2019-03-27T10:42:00.0697522Z     "cloudName": "AzureCloud",
2019-03-27T10:42:00.0698958Z     "id": "88bfee03-551c-4ed3-98b0-be68aee330bb",
2019-03-27T10:42:00.0704752Z     "isDefault": true,
2019-03-27T10:42:00.0705381Z     "name": "Visual Studio Enterprise",
2019-03-27T10:42:00.0706362Z     "state": "Enabled",
2019-03-27T10:42:00.0707434Z     "tenantId": "***",
2019-03-27T10:42:00.0716107Z     "user": {
2019-03-27T10:42:00.0717485Z       "name": "***",
2019-03-27T10:42:00.0718161Z       "type": "servicePrincipal"
2019-03-27T10:42:00.0718675Z     }
2019-03-27T10:42:00.0719185Z   }
2019-03-27T10:42:00.0719831Z ]
2019-03-27T10:42:00.0728173Z [command]/usr/bin/az account set --subscription 88bfee03-551c-4ed3-98b0-be68aee330bb
2019-03-27T10:42:00.8569816Z [command]/bin/bash /home/vsts/work/_temp/azureclitaskscript1553683312219.sh
2019-03-27T10:42:02.4431342Z ERROR: Directory permission is needed for the current user to register the application. For how to configure, please refer 'https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal'. Original error: Insufficient privileges to complete the operation.
2019-03-27T10:42:02.5271752Z [command]/usr/bin/az account clear
2019-03-27T10:42:03.3092558Z ##[error]Script failed with error: Error: /bin/bash failed with return code: 1
2019-03-27T10:42:03.3108490Z ##[section]Finishing: Azure CLI 

Answer 1

Here is how I do it with Azure Pipelines.

1. Create a Service Principal for Terraform.
2. Create the following variables in your pipeline
   • ARM_CLIENT_ID
   • ARM_CLIENT_SECRET
   • ARM_SUBSCRIPTION_ID
   • ARM_TENANT_ID

If you choose to store ARM_CLIENT_SECRET as a secret in Azure DevOps you will need to do the following in your task under the Environment Variables sections of the task to get it decrypted so terraform can read it.

Answer 2

you just need to grant your service connections rights to create service principals. but I'd generally advise against that, just precreate a service principal and use it in your pipeline. creating a new service principal on each run seems excessive.

you can use build\release variables and populate those with client id\secret

Answer 3

The approach defined in the post https://medium.com/@maninder.bindra/creating-a-single-azure-devops-yaml-pipeline-to-provision-multiple-environments-using-terraform-e6d05343cae2? can be considered as well. Here the Keyvault task is used to fetch the secrets from Azure Vault (these include terraform backend access secrets as well as aks sp secrets):

#KEY VAULT TASK
- task: AzureKeyVault@1
  inputs:
    azureSubscription: '$(environment)-sp'
    KeyVaultName: '$(environment)-pipeline-secrets-kv'
    SecretsFilter: 'tf-sp-id,tf-sp-secret,tf-tenant-id,tf-subscription-id,tf-backend-sa-access-key,aks-sp-id,aks-sp-secret'
  displayName: 'Get key vault secrets as pipeline variables'

And then you can use the secrets as variables in the rest of the pipeline. FOr instance aks-sp-id can be referred to as $(aks-sp-id). So the bash/azure-cli task can be something like

# AZ LOGIN USING TERRAFORM SERVICE PRINCIPAL
- script: |
    az login --service-principal -u $(tf-sp-id) -p $(tf-sp-secret) --tenant $(tf-tenant-id)
    cd $(System.DefaultWorkingDirectory)/tf-infra-provision

Followed by terraform init and plan (plan shown below, see post for complete pipeline details)

# TERRAFORM PLAN    
    echo '#######Terraform Plan########'
    terraform plan -var-file=./tf-vars/$(tfvarsFile) -var="client_id=$(tf-sp-id)" -var="client_secret=$(tf-sp-secret)" -var="tenant_id=$(tf-tenant-id)" -var="subscription_id=$(tf-subscription-id)" -var="aks_sp_id=$(aks-sp-id)" -var="aks_sp_secret=$(aks-sp-secret)" -out="out.plan"

Hope this helps.