How can I use Terraform workspace to switch between aws and localstack?

Question

I'd like to use Terraform workspace to manage two environment, one for aws, the other is for localstack. Below is the configuration file of Terraform. I don't want the endpoints and skip_xxx properties on aws workspace. How can I implement that logic? The only way I can think of is to create multiple folders for different environments, such as aws, openstack and shared. But I'd like to find a solution on workspace since it is build-in feature with Terraform for different envs.

provider "aws" {
  profile = "default"
  region  = "ap-southeast-2"
  skip_credentials_validation = true
  skip_metadata_api_check     = true

  endpoints {
       lambda = "http://localhost:4574"
  }
}

Answer 1

The Terraform workspaces feature is for storing multiple states in a single location and switching between them. If you wish to store states in different locations (different S3 buckets, or in your case one not using S3 at all) then the workspaces feature is not appropriate.

Instead, you can use multiple configuration roots and factor the shared configuration out into one or more child modules that both configurations share. Your separate root modules can then represent all of the differences between these targets, including the different backend configurations.

provider "aws" {
  profile = "default"
  region  = "ap-southeast-2"

  skip_credentials_validation = true
  skip_metadata_api_check     = true

  endpoints {
    lambda = "http://localhost:4574"
  }
}

module "shared_something" {
  source = "./modules/shared-something"

  # Configuration for this module that makes sense for localstack
}

provider "aws" {
  profile = "default"
  region  = "ap-southeast-2"
}

module "shared_something" {
  source = "./modules/shared-something"

  # Configuration for this module that makes sense for real AWS
}

If your configuration is relatively simple then having just a single shared module with everything in it may be sufficient, but for more complex situations you may need to use module composition to decompose the system and better support any configuration variations that are needed between AWS and localstack.

Note that the Terraform AWS provider supports non-AWS implementations only on a best-effort basis and may not behave correctly if localstack is not 100% behavior-compatible with AWS. If you are using any third-party modules then they may make assumptions that do not hold for localstack.