Setting up on ISTIO on EKS cluster using Terraform or Helm

Question

I'm new to Terraform and Helm world! I need to set up Istio on the AWS EKS cluster. I was able to set up the EKS cluster using Terraform. I'm thinking of installing ISTIO on top of the EKS cluster using Terraform by writing terraform modules. However, I found that we can set up Istio on top of eks using the helm chart.

Can someone help me to answer my few queries:

1. Should I install Istio using Terraform? If yes, Is there any terraform module available or How can I write one?
2. Should I install Istio using Helm Chart? If yes, what are the pros and cons of it?
3. I need to write a pipeline to install Istio on EKS cluster. Should I use a combination of both terraform and Helm as provider?

Thank you very much for your time. Appreciate all your help!

Answer 1

As @Matt Schuchard mentioned, this is a bit opinion based question, that's why I will answer that based on my understanding.

---

Question number 1.

1. To answer your question, Should I install Istio using Terraform?, yes, if you follow Devops practices then you should write everything in a code, so I would recommend to do that.

2. As per the second part of your question, If yes, Is there any Terraform module available, no, from what I see currently there is no Istio module for Terraform, there is only a helm one.

3. As for the last part of the first question, How can I write Terraform module? I would recommend to start with the Terraform documentation. There is also a tutorial for creating a module.

---

Question number 2.

1. To answer your question, Should I install Istio using Helm Chart?, depends on your use case, you can do it either with helm or istioctl/istio operator.
2. As for the following question, If yes, what are the pros and cons of it? I'm not sure if the current helm chart is production ready, according to Istio documentation, Providing the full configuration in an IstioOperator CR is considered an Istio best practice for production environments, so from what I understand, you should rather use operator than helm. Also worth to note that the helm chart was not used by several versions, if was broughts back to life in version 1.8.

---

Question number 3.

1. As per the last question, I need to write a pipeline to install Istio on EKS cluster. Should I use a combination of both terraform and Helm as provider?, depends, it could be either Terraform and Helm, but from what I see it's also possible to do that with an Terraform and Istio Operator, there is an example. So it's rather up to you to decide which path will you take.

---

I would also recommend to take a look at this reddit thread. You might find few useful comments from the prod environment here, about installing Istio with Terraform.

Answer 2

I have been researching this in the last months and want to add my findings to @Jakob's answer:

First, there is an answer to the pros/cons of the different installation method, so I will not say anything about that: https://istio.io/latest/faq/setup/#install-method-selection Basically all of them can be done with terraform in a certain way.

1. terraform + istioctl with terraform null_resource provider

This is basically the istioctl install -f <file> command. You can create a template file and to the istictl install command with the null_resource provider.

resource "local_file" "setup_istio_config" {
  content = templatefile("${path.module}/istio-operator.tmpl", {
    enableHoldAppUntilProxyStarts = var.hold_app_until_proxy_starts
  })
  filename = "istio-operator.yaml"
}

resource "null_resource" "install_istio" {
  provisioner "local-exec" {
    command = "istioctl install -f \"istio-operator.yaml\" --kubeconfig ../${var.kubeconfig}"
  }
  depends_on = [local_file.setup_istio_config]
}

Pros:

• Very easy setup

Cons:

• How to upgrade using istioctl upgrade -f <file has to be solved somehow
• istioctl must be installed in different versions when handling multiple clusters with different istio versions
• Right istioctl version must be choosen on setup

I guess you can solve the upgrade process somehow, but the hole process is not really "infrastructure as code" enough. I didn't look into it further, because it doesn't seam to be good practice.

2. terraform + istio operator with terraform null_resource provider and kubectl provider

Similar the istio operator setup initializes the operator pod and takes a istio-operator.yml to setup istio for you.

resource "null_resource" "init_operator" {
  provisioner "local-exec" {
    command = "istioctl operator init --kubeconfig ../${var.kubeconfig}"
  }
}

resource "kubectl_manifest" "setup_istio" {
  yaml_body = <<YAML
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
    name: istio-setup
    namespace: istio-system
spec:
  profile: default
  hub: gcr.io/istio-release
  tag: 1.9.2
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
  meshConfig:
    defaultConfig:
      holdApplicationUntilProxyStarts: ${var.hold_app_until_proxy_starts}"
YAML
  depends_on = [null_resource.init_operator]
}

It would be a good idea to wait for some seconds between the init and applying the config.

Here is a good article about doing this with Azure's aks: https://medium.com/@vipinagarwal18/install-istio-on-azure-kubernetes-cluster-using-terraform-214f6d3f611

Pros:

• Easy to setup
• Easy to upgrade istio using the kubectl provider

As long as helm is in alpha, this might be the best approach.

3. terraform + helm with terraform helm provider

Istio provides some charts for the different componentes, when downloading istioctl. Those can be used for installing it with helm.

resource "helm_release" "istio_base" {
  name       = "istio-base"
  chart      = "./manifests/charts/base"
  namespace  = "istio-system"
}

Cons:

• Not ready for production

Bonus

4. istio manifest + helm

Some time ago I've read an article on how to use istio manifest from istioctl manifest generate in combination with helm to install and mange istio. This approach needs some custom code, but it could be done with terraform and the helm provider as well.

Please read: https://karlstoney.com/2021/03/04/ci-for-istio-mesh/index.html

Conclusion

Installing istio with terraform works but seams to be a bit dirty at the moment. Once the helm setup is stable, I guess this would be the best approach. And with the helm provider it can be composed with terraform creation of other resources. Terraform certainly misses an istio provider, but I don't think they will create one in the foreseeable future.