Split terraform tfstate file

5
votes

We started off with one tfstate file, and over time it's grown quite a bit.

Now, it's really slow to plan and I'd now like to split out into several tfstate files (one for our development environment, one for general shared infrastructure, and one per production environment).

Like how it's described at https://charity.wtf/2016/03/30/terraform-vpc-and-why-you-want-a-tfstate-file-per-env/ and Terraform Multiple State Files Best Practice Examples.

Is there any existing tool (built-in or not) to help with this? Sort of like terraform state mv but between tfstates?

3
terraform
I'd be interested in the tool if there is.I tend to split using terraform rm <resource> and terraform import <resource> into another state file. OR copy the state file and drop what you don't need. Depends on how many resources you're managing here – d1ll1nger
Hm, I suppose it's not that bad to script, we'd terraform state list and then loop over with rm + import... – munchybunch
Not too bad, but I guess the gotcha's would be resources that cannot be imported, for example individual routes within route tables (the last time i checked you couldn't anyway) – d1ll1nger

3 Answers

3
votes

terraform state mv has the -state-out flag, where you can define another state to move your resources into.

However, I couldn't get it to work with version 0.11.14, thus I manually cut & pasted the modules from the state file I needed to move to the other state, which worked brilliantly.

EDIT: Here's a workaround, which basically dwonloads both statefiles, moves the states desired, and then reuploads them to the S3 buckets: https://github.com/hashicorp/terraform/pull/15652#issuecomment-410754814

0
votes

Having a separate state file each for infrastructure and application makes sense.

Is there any existing tool (built-in or not) to help with this? Sort of like terraform state mv but between tfstates?

No, as far as I can tell. Move out shared parts (e.g. ECS clusters, ALB, network configuration, iam roles etc.) into a separate project/repository.

When using S3 as state backend you can define different paths for your infrastructure and application states, for example:

  • /infrastructure/nonprod/terraform.tfstate
  • /infrastructure/prod/terraform.tfstate
  • /apps/app1/test/terraform.tfstate
  • /apps/app1/uat/terraform.tfstate
  • /apps/app1/prod/terraform.tfstate

When you want to deploy your application to TEST or UAT you simply call terraform init before terraform apply in your infrastructure project by providing the path to your non-prod S3 state. Then call terraform init on your app terraform config by providing the path to your TEST or UAT path.

Ideally, you can create your own shell scripts to provision and deploy your apps. Then in your favourite CI you can create a pipeline to provision infrastructure and deploy apps as you wish. Make sure you parameterize those scripts so you can pass which environment you want to provision or which app you want to deploy, for example:

./my-shared-infrastructure/provision-infrastructure.sh nonprod

./my-app-1/deploy-application.sh uat v1.0

-2
votes

In a complex world, from what it happens I am part of, we went even further. Following a communal effort we successfully segregated not only the environments but all of the components that are part of the infrastructure.

Digging down, by components I mean a certain set of modules/outputs per component with no extra resources in the configuration but of course the dependencies problem had appeared... nothing that can't be solved with some data{} references to the tiny states and a few scripts.

The result is awesome. We have the same component instantiated with different parameters and each environment which is clean, easy to maintain and operate.

I do not recommend the approach in the case of existing chunky infrastructure where some additional transition related activities are required if you are sick and tired of a single monolithic state file, but it enables a lot of acceleration in a greenfield project.