Deploying helm charts via Terraform Helm provider and Azure DevOps while fetching the helm charts from ACR

Question

I am trying to deploy the helm charts from ACR to an AKS cluster using Terraform helm provider and Azure DevOps container job but it fails while fetching the helm chart from ACR. Please let me know what is going wrong.

helm provider tf module:

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io"
}
# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

provider:

  version                    = "=1.36.0"
  tenant_id                  = var.ARM_TENANT_ID
  subscription_id            = var.ARM_SUBSCRIPTION_ID
  client_id                  = var.ARM_CLIENT_ID
  client_secret              = var.ARM_CLIENT_SECRET
  skip_provider_registration = true
}

data "azurerm_kubernetes_cluster" "aks_cluster" {
  name                = var.aks_cluster
  resource_group_name = var.resource_group_aks
}

locals {
  kubeconfig_path = "/tmp/kubeconfig"
}

resource "local_file" "kubeconfig" {
  filename = local.kubeconfig_path
  content  = data.azurerm_kubernetes_cluster.aks_cluster.kube_admin_config_raw
}

provider "helm" {
  home = "resources/.helm"
  kubernetes {
    load_config_file = true
    config_path = local.kubeconfig_path
  }
}

module "aks_resources" {
  source = "./modules/helm/aks-resources"
}

error: Error: Looks like "" is not a valid chart repository or cannot be reached: Failed to fetch /index.yaml : 404 Not Found

Answer 1

Until now, Helm still doesn't support directly installing chart from an OCI registry.

The recommended steps are:

1. helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
2. helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
3. helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
4. cd install & helm install myhelmtest ./hello-world

So my solution is:

resource "null_resource" "download_chart" {
  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login mycontainerregistry.azurecr.io --username someuser --password somepass
      helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
    EOT
  }
}

resource "helm_release" "chart" {
  name             = "hello_world"
  repository       = "./install"
  chart            = "hello-world"
  version          = "v1"

  depends_on = [null_resource.download_chart]
}

Not perfect but works.

Answer 2

The problem is that you use the wrong url in the Terraform helm_repository. The right url for ACR looks like this:

https://acrName.azurecr.io/helm/v1/repo

And the ACR is a private registry, so it means you need to add the username and password for it. Finally, your Terraform code should like this with version 2.0+ of helm provider:

resource "helm_release" "my-chart" {
  name  = "my-chart"
  chart = "my/chart"
  repository  = "https://${var.acr_name}.azurecr.io/helm/v1/repo"
  repository_username = var.acr_user_name
  repository_password = var.acr_user_password
}

Or with 1.x helm provider:

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io/helm/v1/repo"

  username = "xxxxx"
  password = "xxxxx"
}

# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

Update

Here is the screenshot that it works well and deploy the charts in the AKS:

Answer 3

Small enhancement to the above solution. Include a trigger to force a download of the chart every time. Otherwise, it expects that you always maintain the local copy of the chart post the first deployment

resource "null_resource" "download_chart" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login ${var.registry_fqdn} --username ${var.acr_client_id} --password ${var.acr_client_secret}
      helm chart remove ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart pull ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart export ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag} --destination ./install
    EOT
  }
}