Azure ARM Template Keyvault Resources keeps removing other access policies

Question

I created an ARM template to deploy an Azure WebApp that is using Managed Service Identity authentication with KeyVault for secrets. So the ARM template creates the WebApp resource and enables MSI, and also creates the KeyVault resource and add the WebApp tenantid and objectid to the accessPolicies, however, the ARM template also removes all other existing access policies from my Keyvault.

Is there a way to do an incremental deployment of the access policies, so that I don't have to add back users to the KeyVault access policies after deploymment?

{
  "type": "Microsoft.KeyVault/vaults",
  "name": "[parameters('ICMODSKeyVaultName')]",
  "apiVersion": "2016-10-01",
  "location": "[resourceGroup().location]",
  "properties": {
    "sku": {
      "family": "A",
      "name": "standard"
    },
    "tenantId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').tenantId]",
    "accessPolicies": [
      {
        "tenantId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').tenantId]",
        "objectId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').principalId]",
        "permissions": {
          "secrets": [
            "get"
          ]
        }
      }
    ],
    "enabledForDeployment": true,
    "enabledForTemplateDeployment":  true
  },
  "dependsOn": [
    "[concat('Microsoft.Web/sites/', parameters('AppName'))]"
  ]
}

Answer 1

This is the behavior you should get, as ARM templates are idempotent. you will be able to alter this behavior if you create access policies as a separate resource:

{
  "name": "vaultName/policyName",
  "location": xxx,
  "api-version": "2016-10-01",
  "type": "Microsoft.KeyVault/vaults/accessPolicies",
  "properties": {
    "accessPolicies": [
      {
        "tenantId": "00000000-0000-0000-0000-000000000000",
        "objectId": "00000000-0000-0000-0000-000000000000",
        "permissions": {
          "keys": [
            "encrypt"
          ],
          "secrets": [
            "get"
          ],
          "certificates": [
            "get"
          ]
        }
      }
    ]
  }
}

keep in mind this is a rough sketch, it might not work, but you can get it to work fairly easily. it is to illustrate the idea.

reference: https://docs.microsoft.com/en-us/rest/api/keyvault/vaults/updateaccesspolicy

Answer 2

With the KeyVault's apiVersion 2019-09-01 you can workaround this issue by deploying the vault (type: Microsoft.KeyVault/vaults) only when it's new (using condition).

The access-policies can then be defined separatly as a sub-resource using type Microsoft.KeyVault/vaults/accesspolicies

Answer 3

The issue with the accepted answer is that it removes the key vault from the ARM template altogether, meaning that the key vault's creation becomes a manual process on new environments.

ARM does not allow a key vault to be redeployed without clearing its existing access policies. The accessPolicies property is required (except when recovering a deleted vault), so omitting it will cause an error. Setting it to [] will clear all existing policies. There has been a Microsoft Feedback request to fix this since 2018, currently with 152 votes.

The best way I've found of working around this is to make the key vault deployed conditionally only if it does not already exist, and define the access policies through a separate add child resource. This causes the specified policies to get added or updated, whilst preserving any other existing policies. I check whether the key vault already exists by passing in the list of existing resource names to the ARM template.

In the Azure pipeline:

- task: AzurePowerShell@5
  displayName: 'Get existing resource names'
  inputs:
    azureSubscription: '$(armServiceConnection)'
    azurePowerShellVersion: 'LatestVersion'
    ScriptType: 'InlineScript'
    Inline: |      
      $resourceNames = (Get-AzResource -ResourceGroupName $(resourceGroupName)).Name | ConvertTo-Json -Compress
      Write-Output "##vso[task.setvariable variable=existingResourceNames]$resourceNames"
    azurePowerShellVersion: 'LatestVersion'

- task: AzureResourceManagerTemplateDeployment@3
  name: DeployResourcesTemplate
  displayName: 'Deploy resources through ARM template
  inputs:
    deploymentScope: 'Resource Group'
    action: 'Create Or Update Resource Group'
    # ...
    overrideParameters: >-
      -existingResourceNames $(existingResourceNames)
      # ...
    deploymentMode: 'Incremental'

In the ARM template:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",

  "parameters": {
    "keyVaultName": {
      "type": "string"
    },
    "existingResourceNames": {
      "type": "array",
      "defaultValue": []
    }
  },

  "resources": [
    {
      "type": "Microsoft.KeyVault/vaults",
      "apiVersion": "2016-10-01",
      "name": "[parameters('keyVaultName')]",
      "location": "[resourceGroup().location]",
      // Only deploy the key vault if it does not already exist.
      // Conditional deployment doesn't cascade to child resources, which can be deployed even when their parent isn't.
      "condition": "[not(contains(parameters('existingResourceNames'), parameters('keyVaultName')))]",
      "properties": {
        "sku": {
          "family": "A",
          "name": "Standard"
        },
        "tenantId": "[subscription().tenantId]",
        "enabledForDeployment": false,
        "enabledForDiskEncryption": false,
        "enabledForTemplateDeployment": true,
        "enableSoftDelete": true,
        "accessPolicies": []
      },
      "resources": [
        {
          "type": "accessPolicies",
          "apiVersion": "2016-10-01",
          "name": "add",
          "location": "[resourceGroup().location]",
          "dependsOn": [
            "[parameters('keyVaultName')]"
          ],
          "properties": {
            "accessPolicies": [
              // Specify your access policies here.
              // List does not need to be exhaustive; other existing access policies are preserved.
            ]
          }
        }
      ]
    }
  ]
}

Answer 4

We follow the below workaround solution to do idempotent key vault ARM deployment.

1.Deploying Resource group tags used as checkpoint. In Parameters block

"resourceTags": {
  "type": "object",
  "defaultValue": {
    "DeploymentLabel": "1"
  }
}

In Resource block

{
  "name": "default",
  "type": "Microsoft.Resources/tags",
  "apiVersion": "2020-10-01",
  "properties": {
    "tags": "[parameters('resourceTags')]"
  }
}

2. Now Conditional deployment can be used, if only one tag exist:

this means only Resource group is created and yet to create the Keyvault.

{
  "apiVersion": "2019-09-01",
  "name": "[parameters('keyVaultName')]",
  "location": "[resourceGroup().location]",
  "condition": "[equals(variables('tagslength'), 1)]",
  "type": "Microsoft.KeyVault/vaults",
 }

In the same keyvault arm template, add new tag to register the for Keyvault deployment. Now, there are 2 tags added to RG, so we could redeploy without losing access policies.