How to use terraform to clone a git repo, update files, and push commits to it?

2
votes

Problem

  • We write config files using Terraform for both our Kubernetes Cluster or Apps
  • Some of these files must be pushed to different git repos
    • Just following GitOps for kubernetes and dynamic config repos

Question

So far, I have the following:

  • Generate the configs:
# https://stackguides.com/questions/36629367/getting-an-environment-variable-in-terraform-configuration/36672931#36672931
variable GITLAB_CLONE_TOKEN {}

locals {
  carCrdInstance = {
    apiVersion = "car.io/v1"
    kind       = "Car"
    metadata = {
      name = "super-car"
    }
    spec = {
      convertible = "true"
      color = "black"
    }
  }

  # https://docs.gitlab.com/ee/user/project/deploy_tokens/#git-clone-a-repository
  clone_location = "${path.module}/.gitops"
  branch = "feature/crds-setup"
}

resource "null_resource" "git_clone" {
  provisioner "local-exec" {
    command = "git clone --branch ${local.branch} https://${var.username}:${var.GITLAB_CLONE_TOKEN}@gitlab.example.com/tanuki/awesome_project.git ${local.clone_location}"
  }
}

resource "local_file" "cert_manager_cluster_issuer_object" {
  content  = yamlencode(local.cert_issuer)
  filename = "${git_repo.configs.destination}/crds/instances/white-convertible.yaml"

  # https://stackguides.com/questions/52421656/terraform-execute-script-before-lambda-creation/52422595#52422595
  depends_on = ["null_resource.git_clone"]

  # https://stackguides.com/questions/7149984/how-do-i-execute-a-git-command-without-being-in-the-repository/35899275#35899275
  provisioner "local-exec" {
    command = "git -C ${local.clone_location} commit -am ':new: updating cars...'"
  }

  provisioner "local-exec" {
    command = "git -C ${local.clone_location} push origin ${local.branch}'"
  }
}

Is there anything like that?

  • I haven't tested this above, but I'm looking for something that allows me to do that
1
gitkubernetesterraform
Could you step back a little and explain what you're trying to achieve please? This feels very much like an X Y problem and there might be a much better way to handle this. – ydaetskcoR
Thank you for the question @ydaetskcoR... I intend to use through a GitOps pipeline using ArgoCD... This is to maintain most of the Kubernetes state (System and Application Level). So, I will be generating configs for the various systems that depend on the Cloud-specific host (EKS, Google, Azure) ... Say I create a certificate in AWS; then I need to setup the LB with the ARN of the cert... Having a Kustomize template for all apps with that value, I would just push that the current metadata change to the Github repo ArgoCD syncs all my apps from (versioned way). Does it make sense now? – Marcello de Sales
There are other applications such as updating a CRD with those values, which in turn can mutate any K8s object depending on its type, permissions, etc... It's just a natural flow and separation of what needs to be setup by Terraform and what needs to be setup by regular Kubernetes and where the values come from. A more decoupled way to avoid Terraform dealing with the actual k8s objects, but rather creating needed data for ArgoCD to update. – Marcello de Sales
@MarcellodeSales Did you ever find a solution that was to your liking? I'd be interested to know. – siwyd

1 Answers

1
votes

How can I perform a git clone, commit, push using terraform?

Should we just use shell?

Terraform is a good tool - it is best for provisioning immutable infrastructure. Shell script might also have its place, but when you can, it is preferably to use a more declarative approach.

What you describe with "git clone, commit, push" is essentially some of the steps that is commonly done in something like a Build or Deployment Pipeline. Terraform might be a good tool to use in some of the steps, but it is not the best tool to orchestrate the full workflow, in my point of view.

A tool made for orchestrating pipeline workflows might be best for this, like e.g.