Background:
I'm using an AWS CodeBuild buildspec.yml to iterate through directories from a GitHub repo to apply IaC using Terraform. To access the credentials needed for the Terraform AWS provider, I used AWS system manager parameter store to retrieve the access and secret key within the buildspec.yml.
Problem:
The system manager parameter store masks the access and secret key env value so when they are inherited by the Terraform AWS provider, the provider outputs that the credentials are invalid:
Error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
status code: 403, request id: xxxx
To reproduce the problem:
Create system manager parameter store variables (TF_VAR_AWS_ACCESS_KEY_ID=access, TF_AWS_SECRET_ACCESS_KEY=secret)
Create AWS CodeBuild project with:
"source": {
"type": "NO_SOURCE",
}
"environment": {
"type": "LINUX_CONTAINER",
"image": "aws/codebuild/standard:4.0",
"computeType": "BUILD_GENERAL1_SMALL"
}
buildspec.yml
with the following: (modified to create .tf files instead of sourcing from github)
version: 0.2
env:
shell: bash
parameter-store:
TF_VAR_AWS_ACCESS_KEY_ID: TF_AWS_ACCESS_KEY_ID
TF_VAR_AWS_SECRET_ACCESS_KEY: TF_AWS_SECRET_ACCESS_KEY
phases:
install:
commands:
- wget https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip -q
- unzip terraform_0.12.28_linux_amd64.zip && mv terraform /usr/local/bin/
- printf "provider "aws" {\n\taccess_key = var.AWS_ACCESS_KEY_ID\n\tsecret_key = var.AWS_SECRET_ACCESS_KEY\n\tversion = \"~> 3.2.0\"\n}" >> provider.tf
- printf "variable "AWS_ACCESS_KEY_ID" {}\nvariable "AWS_SECRET_ACCESS_KEY" {}" > vars.tf
- printf "resource \"aws_s3_bucket\" \"test\" {\n\tbucket = \"test\"\n\tacl = \"private\"\n}" >> s3.tf
- terraform init
- terraform plan
Attempts:
- Passing creds through terraform
-vars
option:
terraform plan -var="AWS_ACCESS_KEY_ID=$TF_VAR_AWS_ACCESS_KEY_ID" -var="AWS_ACCESS_KEY_ID=$TF_VAR_AWS_SECRET_ACCESS_KEY"
but I get the same invalid credentials error
- Export system manager parameter store credentials within buildspec.yml:
commands:
- export AWS_ACCESS_KEY_ID=$TF_VAR_AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$TF_VAR_AWS_SECRET_ACCESS_KEY
which results in duplicate masked variables and the same error above. printenv
output within buildspec.yml:
AWS_ACCESS_KEY_ID=***
TF_VAR_AWS_ACCESS_KEY_ID=***
AWS_SECRET_ACCESS_KEY=***
TF_VAR_AWS_SECRET_ACCESS_KEY=***
Possible solution routes:
- Somehow pass the MASKED parameter store credential values into Terraform successfully (preferred)
- Pass sensitive credentials into the Terraform AWS provider using a different method e.g. AWS secret manager, IAM role, etc.
- Unmask the parameter store variables to pass into the aws provider (probably defeats the purpose of using aws system manager in the first place)