Terraform provider/variable sharing in modules

31
votes

Is there a way of abstracting the provider for all the modules defined in a project.

for example, I have this project

├── modules
│   ├── RDS
│   └── VPC
└── stacks
    ├── production
    │   └── main.tf
    └── staging
        └── main.tf

and it works fine... the problem is with the definition of modules

├── RDS
│   ├── README.md
│   ├── main.tf
│   ├── providers.tf
│   └── variables.tf
└── VPC
    ├── README.md
    ├── main.tf
    ├── providers.tf
    └── variables.tf

the provider in both of these modules are exactly the same

# providers.tf
provider "aws" {
  region = "${var.region}"
  version = "~> 1.26"
}

and the variables in each module are different but they all have the region variable.

# variables.tf
variable "region" {
  default     = "eu-central-1"
  description = "AWS region."
}
# other module dependent variables...

is there a way to define those bits of information on the modules level so that I end up with something roughly like this

├── modules
│   ├── providers.tf  <<< include the *shared* provider definition block
│   ├── variables.tf  <<< include the *shared* region vaiable definition block
│   ├── RDS
│   │   ├── README.md
│   │   ├── main.tf
│   │   └── variables.tf
│   └── VPC
│       ├── README.md
│       ├── main.tf
│       └── variables.tf

one last thing, the modules definitions most of the time have a resource attribute (pulling a module from the terraform registry... therefore I don't know if it's feasible to inherit both the source from the registry and a base module)

4
terraformterraform-provider-aws
You should use workspaces/branches instead of placing the environments in a directory. That will solve your problem and follow best practices. terraform.io/docs/enterprise/workspaces/repo-structure.html – Matt Schuchard
I went with the simplest solution to symlink the providers.tf file... I'm planning to test terragrunt soon and see how it goes... right now symlinks works fine... as for the workspace branches model it's a bit complicated (and the team won't benefit from it since we are all new to terraform) and will just tangle ourselves in the branches when trying to fix something (that needs to be applied in many branches) – a14m
@MattSchuchard That terraform link gives three options, and one is to use a branch per environment. For typical git-based repositories, using branching in such a way goes against the most base git guidance. – GaTechThomas

4 Answers

13
votes

Right now it's not possible to achieve that. There were previous discussions on github about the same topic in the following issues:

TL;DR
the sharing of variables between modules is against terraform core clarity/explicity principles.

Workaround
A workaround is to have the *shared* files in the parent directory and using symlinks to add them to the modules.

6
votes

If you know terragrunt, this will be no problem at all.

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules.

It is designed for the problem you just stuck with.

account
 └ _global
 └ region
    └ _global
    └ environment
       └ resource

Quick start

Check out the terragrunt-infrastructure-modules-example and terragrunt-infrastructure-live-example repos for fully-working sample code that demonstrates these features

you can use prod/terraform.tfvars or prod/account.tfvars for global variables or put the tfvars file under _global folder.

5
votes

You can abstract provider parameters away from a module by passing in a provider alias to use. This allows you to create a module without reference to things like Region and then pass those details in on invocation.

For your use case, you can define aliased providers in your stack folders (probably best to define this in a file and make symbolic links for each stack folder):

# stacks/{staging,production}/providers.tf
provider "aws" {
  alias  = "us-east-1"
  region = "us-east-1"
}

provider "aws" {
  alias   = "us-east-2"
  region  = "us-east-2"
}

Then when you invoke the modules, pass in the provider alias you want to use (this assumes a module uses only 1 of any particular provider type):

# stacks/{staging,production}/main.tf
module "VPC-us-east-1" {
  source = "../../modules/VPC"

  providers = {
    aws      = "aws.us-east-1"
  }
}

module "VPC-us-east-2" {
  source = "../../modules/VPC"

  providers = {
    aws      = "aws.us-east-2"
  }
}
0
votes

Currently there is no global variable feature for Hashicorp configuration language.

And variables are used for defining module interfaces of modules or act as like ENV variables (inline ENV). So, it's not the best way to use variable as globally shared. Because each time you define some modules you always need to think about which variables you used on upper modules. And probably you will end up like: first_module_s3_bucket_name, second_module_s3_bucket_name.

But locals are more makes sense to be shared to child modules. Because, locals suppose to help avoid repeat same value.

There is a new discussion on Hashicorp feature request if you interested.

https://github.com/hashicorp/terraform/issues/25431