Terraform profile field usage in AWS provider

Question

I have a $HOME/.aws/credentials file like this:

[config1]
aws_access_key_id=accessKeyId1
aws_secret_access_key=secretAccesskey1

[config2]
aws_access_key_id=accessKeyId2
aws_secret_access_key=secretAccesskey2

So I was expecting that with this configuration, terraform will choose the second credentials:

terraform {
  backend "s3" {
    bucket  = "myBucket"
    region  = "eu-central-1"
    key     = "path/to/terraform.tfstate"
    encrypt = true
  }
}

provider "aws" {
  profile = "config2"
  region  = "eu-central-1"
}

But when I try terraform init it says it hasn't found any valid credentials:

Initializing the backend...

Error: No valid credential sources found for AWS Provider. Please see https://terraform.io/docs/providers/aws/index.html for more information on providing credentials for the AWS Provider

As as workaround, I changed config2 by default in my credentials file and I removed the profile field from the provider block so it works but I really need to use something like the first approach. What am I missing here?

How are you configuring your remote state? And can you share the exact error? — ydaetskcoR

ydaetskcoR ydaetskcoR · Accepted Answer · 2019-07-07T17:36:09

Unfortunately you also need to provide the IAM credential configuration to the backend configuration as well as your AWS provider configuration.

The S3 backend configuration takes the same parameters here as the AWS provider so you can specify the backend configuration like this:

terraform {
  backend "s3" {
    bucket  = "myBucket"
    region  = "eu-central-1"
    key     = "path/to/terraform.tfstate"
    encrypt = true
    profile = "config2"
  }
}

provider "aws" {
  profile = "config2"
  region  = "eu-central-1"
}

There's a few reasons behind this needing to be done separately. One of the reasons would be that you can independently use different IAM credentials, accounts and regions for the S3 bucket and the resources you will be managing with the AWS provider. You might also want to use S3 as a backend even if you are creating resources in another cloud provider or not using a cloud provider at all, Terraform can manage resources in a lot of places that don't have a way to store Terraform state. The main reason though is that the backends are actually managed by the core Terraform binary rather than the provider binaries and the backend initialisation happens before pretty much anything else.

Terraform profile field usage in AWS provider

1 Answers