Modularising Terraform IaC across microservice environments

Question

I am trying to refactor my IaC Terraform set up to repeat less code and be quicker to make changes. I am working on a serverless microservice application, and so, for example i am running a few instances of aws-ecs-autoscaling and aws-ecs. I have develop and production environments, and within each one a modules folder where each microservice module is defined. Please see image for mock folder structure.

As you can see there are many repeated folders. In the main.tf of the dev and prod environments each module is called and vars assigned.

EG:

ecs-autoscaling-microservice-A main.tf

resource "aws_appautoscaling_target" "dev_ecs_autoscaling_microservice_A_target" {
  max_capacity = 2
  min_capacity = 1
  resource_id = "service/${var.ecs_cluster.name}/${var.ecs_service.name}"
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace = "ecs"
}

resource "aws_appautoscaling_policy" "dev_ecs_autoscaling_microservice_A_memory" {
  name               = "dev_ecs_autoscaling_microservice_A_memory"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.resource_id
  scalable_dimension = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.scalable_dimension
  service_namespace  = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.service_namespace

  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ECSServiceAverageMemoryUtilization"
    }
    target_value       = 80
  }
}

resource "aws_appautoscaling_policy" "dev_ecs_autoscaling_microservice_A_cpu" {
  name = "dev_ecs_autoscaling_microservice_A_cpu"
  policy_type = "TargetTrackingScaling"
  resource_id = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.resource_id
  scalable_dimension = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.scalable_dimension
  service_namespace = aws_appautoscaling_target.dev_ecs_autoscaling_microservice_A_target.service_namespace

  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ECSServiceAverageCPUUtilization"
    }
    target_value = 60
  }
}

DEVELOP main.tf

module "ecs_autoscaling_microservice_A" {
  source      = "./modules/ecs-autoscaling-microservice-A"
  ecs_cluster = module.ecs_autoscaling_microservice_A.ecs_cluster_A
  ecs_service = module.ecs_autoscaling_microservice_A.ecs_service_A
}

My question is what is the best way to go about removing all modules. SO that instead of having an ecs module for each microservice for both prod and dev environments, I can just have 1 module for ecs, that can be re used for any microservice in any environment. See Image for required folder structure. Is this possible or am i wasting my time? I was thinking of using some kind of for_each where each microservice is defined before hand with its owned mapped variables. But would like some guidance please. Thanks in advance!

You would have to parameterize your modules. You could also use tf workspaces for dev and prod environments. — Marcin

dennyrolling dennyrolling · Accepted Answer · 2021-04-08T16:55:53

I suggest you read the excellent series of blog posts on Terraform by Yevgeniy Brikman which cleared my understanding of Terraform:

https://blog.gruntwork.io/a-comprehensive-guide-to-terraform-b3d32832baca

This exact question seems to be touched on in this one: https://blog.gruntwork.io/how-to-create-reusable-infrastructure-with-terraform-modules-25526d65f73d

Modularising Terraform IaC across microservice environments

1 Answers