Terraform best pracitcies - Avoid duplication

Question

I am considering switching some of our infrastructure code over to Terraform, as we hit a bit of a wall with AWS CloudFormation.

Let me take an example of what I'm trying to achieve. I have an ECS cluster which can run around 10+ different task definitions. Each task definition contains almost the same configurations so to avoid code duplication I am building a reusable module.

From what I have gathered there are 3 primary ways to structure your terraform directories.

1:

project
├── modules
|      ├─ ecs
|            ├── main.tf
|            ├── variables.tf
|            ├── task
|                 ├── main.tf
|                 ├── variables.tf
|
|
└── env
|       ├─── dev.tfvars
|       ├─── prod.tfvars
|       ├─── stage.tfvars
|       ├─── 10+   
|
|
|── main.tf 
|── variable.tf

With this structure, I would have to get the variables from the env folder and pass it with -var-files="env/dev.tfvars" and then in my main.tf I would have to pass the variable to the modules/ecs/task/main.tf which seems like a long way, and a lot of steps just to get a variable to the ecs/task/main.tf

2.

project
├── modules
|       ├─ ecs
|            ├── main.tf
|            ├── variables.tf
|            ├── task
|                ├── main.tf
|                ├── variables.tf
|
|
└── stage
|    ├─── main.tf    
|    ├─── variable.tf
|    ├─── stage.tfvars 
|
└── dev
|    ├─── main.tf    
|    ├─── variable.tf
|    ├─── dev.tfvars
|
└── 10+
|   ├─── main.tf    
|   ├─── variable.tf
|   ├─── X.tfvars

If you wish to apply environment-based variables to the modules/ecs/task/main.tf, you would have to start from the main.tf in for example stage which calls the modules/ecs/main.tf. and from there apply it through the /modules/ecs/main.tf which then applies it to the /modules/ecs/task/main.tf.

The problem with this approach is also whenever I add a new module, I would have to add it to all the different environments' main.tf

3.

project
├── modules
|         ├── ecs
|       ├── main.tf
|       ├── variables.tf
|       ├── task
|            ├── main.tf
|                    ├── variables.tf
|
|
|
|
|──  main.tf 
|── variable.tf

Using terraform workspaces, I can use locals in the modules/ecs/task/variables.tf to determine what environment I am building. Like this:

modules/ecs/task.variables.tf

locals {

env="${terraform.workspace}"

masterAccountIDS = {

"default"="12121212"

"dev"="84848484"

}



masterAccountID="${lookup(local.masterAccountIDS, local.env)}"

}

But this would require that every time I add a new environment I go through all my variables.tf files, and add a new entry called for example "stage".

I can't figure out a way that would allow me not to copy paste code, or insert new things when adding new environments or keep it all located at one place, so I would maybe only have to edit one file.

vivekyad4v vivekyad4v · Accepted Answer · 2018-10-31T02:51:46

Using workspaces is the best bet for now. I am using something like below to manage all variables at one place -

defaults.tf -

/* DEFAULT VARIABLES */

locals {
  tags = {
      Project = "${var.project}"
      Contact = "[email protected]"
      Requester = "Vivek"
      Creator = "Vivek"
      ManagedBy = "TF"
      Environ = "${local.workspace["environ"]}"
   }
}

locals {
  meta = {
      name_prefix = "hpy-${local.workspace["environ"]}-${local.workspace["project_name"]}"
      account_id = "${local.workspace["account_id"]}"
      region_name = "${local.workspace["region_name"]}"
   }
}

/* CUSTOM VARIABLES - ENVIRONMENT SPECIFIC */

################  DEFAULT LOCALS - DEFINE ENV VARIABLES FOR APP  #################
locals {
  env = {
    default = {
      project_name = "${var.project}"
      region_name = "${var.region}"
      environ = "dev"
      account_id = "356******001"
    }
    default_list = {
    }

################  DEV LOCALS - DEFINE ENV VARIABLES FOR APP  #################

    dev = {
      environ = "dev"
    }
    dev_list = {
    }

################  UAT LOCALS - DEFINE ENV VARIABLES FOR APP  #################

    uat = {
      environ = "uat"
    }
    uat_list = {
    }

################  PREP LOCALS - DEFINE ENV VARIABLES FOR APP  #################

    prep = {
      environ = "prep"
    }
    prep_list = {
    }

################  PRD LOCALS - DEFINE ENV VARIABLES FOR APP  #################

    prod = {
      environ = "prod"
      account_id = "3621****8334"
    }
    prod_list = {
    }

  }

################  EXPORTING ENV/WORKSPACE VARIABLES FOR APP  #################

  workspace = "${merge(local.env["default"], local.env[terraform.workspace])}"
  workspace_lists= "${merge(local.env["default_list"], local.env[format("%v_list",terraform.workspace)])}"
}

Now you can call above variables in your main definition -

main.tf -

module "s3" {
  source = "s3"
  tags = "${local.tags}"
  meta = "${local.meta}"
  workspace = "${local.workspace}"
  workspace_list = "${local.workspace_list}"
}

Now you can call defined the variables inside modules as below -

"${var.workspace["environ"]}"

PS - This is a workaround but works really well as if now. Terraform should provide something out of the box to manage it.

Terraform best pracitcies - Avoid duplication

1 Answers