Terraform - How to Create a GKE Cluster and Install Helm Charts?

2
votes

Goal

I have a specific workflow to set up a fresh Kubernetes cluster on Google Cloud. And I want to automate the process with Terraform. Those are the steps:

  1. Create cluster 
    gcloud beta container --project "my-google-project" clusters create "cluster-name" --zone "europe-west3-b"
  2. Setup Helm repos 
    helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo add jetstack https://charts.jetstack.io/
helm repo update
  3. Install NGINX Ingress 
    kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)
helm install nginx-ingress stable/nginx-ingress
  4. Install Cert-Manager 
    kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/v0.13.0/deploy/manifests/00-crds.yaml
kubectl create namespace cert-manager
helm install cert-manager jetstack/cert-manager --namespace cert-manager

Ideas

The first step will probably look like this:

resource "google_container_cluster" "primary" {
  name               = "cluster-name"
  location           = "europe-west3-b"
  initial_node_count = 3

  master_auth {
    username = ""
    password = ""

    client_certificate_config {
      issue_client_certificate = false
    }
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]

    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

But I have no idea how to approach steps 2 - 4.

1
terraformgoogle-kubernetes-enginekubernetes-helmkubectlnginx-ingress
Steps 2-4 are software provisioning and should be performed with a software provisioner instead. The most popular tool for that is Ansible. – Matt Schuchard

1 Answers

3
votes

While Terraform makes sense for building and provisioning cloud infrastructure for things like Kubernetes to run on, it doesn't necessarily make sense to be used to configure said infrastructure after deployment. I think most infrastructure designs would consider applications deployed onto a provisioned cluster as configurations to said cluster. The semantics here are surely a bit nuanced but I maintain that a tool like Ansible is better suited to deploy applications to your cluster after provisioning.

So my advice would be to define a handful of Ansible Roles. Perhaps:

create_cluster
deploy_helm
install_nginx_ingress
install_cert_manager

Within each respective role, define the tasks and variables that are required to be used as per the Galaxy schema. Lastly, define a Playbook that Ansible uses to include or import these roles. This would allow you to provision your infrastructure and deploy all of the required applications to it in a single command:

ansible-playbook playbook.yml