Terraform - How to attach SSL certificate stored in Azure KeyVault to an Application Gateway

Question

I have a Terraform script that create an Azure Key Vault, imports my SSL certificate (3DES .pfx file with a password), and creates an Application Gateway with a HTTP listener. I'm trying to change this to a HTTPS listener that uses my SSL certificate from KeyVault.

I've stepped through this process manually in Azure Portal and I have this working with PowerShell. Unfortunately I don't find Terraform's documentation clear on how this is supposed to be achieved.

Here are relevant snippets of my Application Gateway and certificate resources:

resource "azurerm_application_gateway" "appgw" {
  name                = "my-appgw"
  location            = "australiaeast"
  resource_group_name = "my-rg"
  
  http_listener {
    protocol                       = "https"
    ssl_certificate_name           = "appgw-listener-cert"
    ...
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.appgw_uaid.id]
  }

  ssl_certificate {
    key_vault_secret_id = azurerm_key_vault_certificate.ssl_cert.secret_id
    name                = "appgw-listener-cert"
  }

  ...
}

resource "azurerm_key_vault" "kv" {
  name                       = "my-kv"
  location                   = "australiaeast"
  resource_group_name        = "my-rg"
  ...
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.uaid_appgw.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

resource "azurerm_key_vault_certificate" "ssl_cert" {
  name         = "my-ssl-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate {
    # These are stored as sensitive variables in Terraform Cloud
    # ssl_cert_b64 value was retrieved by: $ cat my-ssl-cert.pfx | base64 > o.txt
    contents = var.ssl_cert_b64
    password = var.ssl_cert_passwd
  }

  certificate_policy {
    issuer_parameters {
      name = "Unknown"
    }

    key_properties {
      exportable = false
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = false
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }
  }
}

Here is the (sanitised) error I get in Terraform Cloud:

Error: waiting for create/update of Application Gateway: (Name "my-appgw" / Resource Group "my-rg"): Code="ApplicationGatewayKeyVaultSecretException" Message="Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/1324/resourceGroups/my-rg/providers/Microsoft.Network/applicationGateways/my-appgw'. See details below:" Details=[{"code":"ApplicationGatewaySslCertificateDoesNotHavePrivateKey","message":"Certificate /subscriptions/1324/resourceGroups/my-rg/providers/Microsoft.Network/applicationGateways/my-appgw/sslCertificates/appgw-listener-cert does not have Private Key."}]

I downloaded the certificate from Key Vault and it appears to be a valid, not corrupted or otherwise broken. I don't understand why the error says it doesn't have a Private Key.

Can someone point out what I've missed or I'm doing wrong?

Answer 1

The issue is that there isn't any access policy defined for the app gateway in the keyvault for which it not able to get the certififcate.

So, inorder to resolve this , you have to add an acess policy for the managed identity that is being used by the application gateway. So, after creating managed identity and before using in application gateway, you have to use something like below:

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = "yourresourcegroup"
  location            = "resourcegrouplocation"
  name                = "mi-appgw-keyvault"
}

data "azurerm_key_vault" "example"{
    name = "testansumankeyvault-01"
    resource_group_name = "yourresourcegroup"
} 
resource "azurerm_key_vault_access_policy" "example" {
  key_vault_id = data.azurerm_key_vault.example.id
  tenant_id    = data.azurerm_client_config.current.tenant_id
  object_id    = azurerm_user_assigned_identity.base.principal_id

  key_permissions = [
    "Get",
  ]

  certificate_permissions = [
      "Get",
  ]

  secret_permissions = [
    "Get",
  ]
}

---

So , Only after the above is done you can use something similar to below as per your requirement:

data "azurerm_user_assigned_identity" "example" {
  name                = "mi-appgw-keyvault"
  resource_group_name = "yourresourcegroup"
}
data "azurerm_key_vault" "example"{
    name = "testansumankeyvault-01"
    resource_group_name = "yourresourcegroup"
} 
resource "azurerm_application_gateway" "appgw" {
  name                = "my-appgw"
  location            = "australiaeast"
  resource_group_name = "my-rg"
  
  http_listener {
    protocol                       = "https"
    ssl_certificate_name           = "appgw-listener-cert"
    ...
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [data.azurerm_user_assigned_identity.example.id]
  }

  ssl_certificate {
    key_vault_secret_id = azurerm_key_vault_certificate.ssl_cert.secret_id
    name                = "appgw-listener-cert"
  }

  ...
}
    
data "azurerm_key_vault_certificate" "example" {
  name         = "secret-sauce"
  key_vault_id = data.azurerm_key_vault.example.id
}

Note:

I have used exisitng keyvault to set the keyvault access policy for testing and also a exisitng certificate in keyvault. If you are creating new then please use 2 deployments:

1. Deploy Keyvault,managed_identity ,access policy and certificate for keyvault first.
2. Then use data sources for keyvault, managed identity and certificate and then deploy the application gateway with ssl certificate referencing from keyvault.

Answer 2

I tested 2 scenarios in my environment :

Scenario 1: Generating a new certificate in Keyvault and uploading it in application gateway ssl certificate.

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

data "azurerm_resource_group" "example"{
    name = "ansumantest"
}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  name                = "mi-appgw-keyvault"
}


resource "azurerm_key_vault" "kv" {
  name                       = "ansumankeyvault01"
  location                   = data.azurerm_resource_group.example.location
  resource_group_name        = data.azurerm_resource_group.example.name
  tenant_id = data.azurerm_client_config.current.tenant_id
  sku_name = "standard"
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.base.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

output "secret_identifier" {
  value = azurerm_key_vault_certificate.example.secret_id
}

resource "azurerm_key_vault_certificate" "example" {
  name         = "generated-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate_policy {
    issuer_parameters {
      name = "Self"
    }

    key_properties {
      exportable = true
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = true
    }

    lifetime_action {
      action {
        action_type = "AutoRenew"
      }

      trigger {
        days_before_expiry = 30
      }
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }

    x509_certificate_properties {
      # Server Authentication = 1.3.6.1.5.5.7.3.1
      # Client Authentication = 1.3.6.1.5.5.7.3.2
      extended_key_usage = ["1.3.6.1.5.5.7.3.1"]

      key_usage = [
        "cRLSign",
        "dataEncipherment",
        "digitalSignature",
        "keyAgreement",
        "keyCertSign",
        "keyEncipherment",
      ]

      subject_alternative_names {
        dns_names = ["internal.contoso.com", "domain.hello.world"]
      }

      subject            = "CN=hello-world"
      validity_in_months = 12
    }
  }
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  address_space       = ["10.254.0.0/16"]
}

resource "azurerm_subnet" "frontend" {
  name                 = "frontend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.0.0/24"]
}

resource "azurerm_subnet" "backend" {
  name                 = "backend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.2.0/24"]
}

resource "azurerm_public_ip" "example" {
  name                = "example-pip"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  allocation_method   = "Static"
  sku = "standard"
}

# since these variables are re-used - a locals block makes this more maintainable
locals {
  backend_address_pool_name      = "${azurerm_virtual_network.example.name}-beap"
  frontend_port_name             = "${azurerm_virtual_network.example.name}-feport"
  frontend_ip_configuration_name = "${azurerm_virtual_network.example.name}-feip"
  http_setting_name              = "${azurerm_virtual_network.example.name}-be-htst"
  listener_name                  = "${azurerm_virtual_network.example.name}-httplstn"
  request_routing_rule_name      = "${azurerm_virtual_network.example.name}-rqrt"
  redirect_configuration_name    = "${azurerm_virtual_network.example.name}-rdrcfg"

}

resource "null_resource" "previous" {}

resource "time_sleep" "wait_240_seconds" {
  depends_on = [azurerm_key_vault.kv]

  create_duration = "240s"
}

resource "azurerm_application_gateway" "network" {
  name                = "example-appgateway"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location

  sku {
    name     = "Standard_v2"
    tier     = "Standard_v2"
    capacity = 2
  }

  gateway_ip_configuration {
    name      = "my-gateway-ip-configuration"
    subnet_id = azurerm_subnet.frontend.id
  }

  frontend_port {
    name = local.frontend_port_name
    port = 443
  }

  frontend_ip_configuration {
    name                 = local.frontend_ip_configuration_name
    public_ip_address_id = azurerm_public_ip.example.id
  }

  backend_address_pool {
    name = local.backend_address_pool_name
  }

  backend_http_settings {
    name                  = local.http_setting_name
    cookie_based_affinity = "Disabled"
    path                  = "/path1/"
    port                  = 443
    protocol              = "Https"
    request_timeout       = 60
  }

  http_listener {
    name                           = local.listener_name
    frontend_ip_configuration_name = local.frontend_ip_configuration_name
    frontend_port_name             = local.frontend_port_name
    protocol                       = "Https"
    ssl_certificate_name = "app_listener"
  }

  identity {
    type = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.base.id]
  }

  ssl_certificate {
    name = "app_listener"
    key_vault_secret_id = azurerm_key_vault_certificate.example.secret_id
  }

  request_routing_rule {
    name                       = local.request_routing_rule_name
    rule_type                  = "Basic"
    http_listener_name         = local.listener_name
    backend_address_pool_name  = local.backend_address_pool_name
    backend_http_settings_name = local.http_setting_name
  }
  depends_on = [time_sleep.wait_240_seconds]
}

Output:

Scenario 2 : Using one certificate which I import from local machine to keyvault and using it in application gateway.

provider "azurerm" {
    features{}
}
data "azurerm_client_config" "current" {}

data "azurerm_resource_group" "example"{
    name = "ansumantest"
}

resource "azurerm_user_assigned_identity" "base" {
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  name                = "mi-appgw-keyvault"
}


resource "azurerm_key_vault" "kv" {
  name                       = "ansumankeyvault01"
  location                   = data.azurerm_resource_group.example.location
  resource_group_name        = data.azurerm_resource_group.example.name
  tenant_id = data.azurerm_client_config.current.tenant_id
  sku_name = "standard"
  access_policy {
    object_id    = data.azurerm_client_config.current.object_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    certificate_permissions = [
      "Create",
      "Delete",
      "DeleteIssuers",
      "Get",
      "GetIssuers",
      "Import",
      "List",
      "ListIssuers",
      "ManageContacts",
      "ManageIssuers",
      "Purge",
      "SetIssuers",
      "Update"
    ]

    key_permissions = [
      "Backup",
      "Create",
      "Decrypt",
      "Delete",
      "Encrypt",
      "Get",
      "Import",
      "List",
      "Purge",
      "Recover",
      "Restore",
      "Sign",
      "UnwrapKey",
      "Update",
      "Verify",
      "WrapKey"
    ]

    secret_permissions = [
      "Backup",
      "Delete",
      "Get",
      "List",
      "Purge",
      "Restore",
      "Restore",
      "Set"
    ]
  }

  access_policy {
    object_id    = azurerm_user_assigned_identity.base.principal_id
    tenant_id    = data.azurerm_client_config.current.tenant_id

    secret_permissions = [
      "Get"
    ]
  }
}

output "secret_identifier" {
  value = azurerm_key_vault_certificate.example.secret_id
}

resource "azurerm_key_vault_certificate" "example" {
  name         = "imported-cert"
  key_vault_id = azurerm_key_vault.kv.id

  certificate {
    contents = filebase64("C:/appgwlistner.pfx")
    password = "password"
  }

  certificate_policy {
    issuer_parameters {
      name = "Self"
    }

    key_properties {
      exportable = true
      key_size   = 2048
      key_type   = "RSA"
      reuse_key  = false
    }

    secret_properties {
      content_type = "application/x-pkcs12"
    }
  }
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  address_space       = ["10.254.0.0/16"]
}

resource "azurerm_subnet" "frontend" {
  name                 = "frontend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.0.0/24"]
}

resource "azurerm_subnet" "backend" {
  name                 = "backend"
  resource_group_name  = data.azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.254.2.0/24"]
}

resource "azurerm_public_ip" "example" {
  name                = "example-pip"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  allocation_method   = "Static"
  sku = "standard"
}

# since these variables are re-used - a locals block makes this more maintainable
locals {
  backend_address_pool_name      = "${azurerm_virtual_network.example.name}-beap"
  frontend_port_name             = "${azurerm_virtual_network.example.name}-feport"
  frontend_ip_configuration_name = "${azurerm_virtual_network.example.name}-feip"
  http_setting_name              = "${azurerm_virtual_network.example.name}-be-htst"
  listener_name                  = "${azurerm_virtual_network.example.name}-httplstn"
  request_routing_rule_name      = "${azurerm_virtual_network.example.name}-rqrt"
  redirect_configuration_name    = "${azurerm_virtual_network.example.name}-rdrcfg"

}

resource "null_resource" "previous" {}

resource "time_sleep" "wait_240_seconds" {
  depends_on = [azurerm_key_vault.kv]

  create_duration = "240s"
}

resource "azurerm_application_gateway" "network" {
  name                = "example-appgateway"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location

  sku {
    name     = "Standard_v2"
    tier     = "Standard_v2"
    capacity = 2
  }

  gateway_ip_configuration {
    name      = "my-gateway-ip-configuration"
    subnet_id = azurerm_subnet.frontend.id
  }

  frontend_port {
    name = local.frontend_port_name
    port = 443
  }

  frontend_ip_configuration {
    name                 = local.frontend_ip_configuration_name
    public_ip_address_id = azurerm_public_ip.example.id
  }

  backend_address_pool {
    name = local.backend_address_pool_name
  }

  backend_http_settings {
    name                  = local.http_setting_name
    cookie_based_affinity = "Disabled"
    path                  = "/path1/"
    port                  = 443
    protocol              = "Https"
    request_timeout       = 60
  }

  http_listener {
    name                           = local.listener_name
    frontend_ip_configuration_name = local.frontend_ip_configuration_name
    frontend_port_name             = local.frontend_port_name
    protocol                       = "Https"
    ssl_certificate_name = "app_listener"
  }

  identity {
    type = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.base.id]
  }

  ssl_certificate {
    name = "app_listener"
    key_vault_secret_id = azurerm_key_vault_certificate.example.secret_id
  }

  request_routing_rule {
    name                       = local.request_routing_rule_name
    rule_type                  = "Basic"
    http_listener_name         = local.listener_name
    backend_address_pool_name  = local.backend_address_pool_name
    backend_http_settings_name = local.http_setting_name
  }
  depends_on = [time_sleep.wait_240_seconds]
}

Outputs:

Note:

Please make sure to have the pfx certificate with private keys. While you are exporting a pfx certificate using a security certificate, please make sure to have the following propeties selected as shown below and then give a password and export it.