Terraform unable to assume roles with MFA enabled

Question

I'm having a terrible time getting Terraform to assume an IAM role with another account with MFA required. Here's my setup

AWS Config

[default]
region = us-west-2
output = json

[profile GEHC-000]
region = us-west-2
output = json

....

[profile GEHC-056]
source_profile = GEHC-000
role_arn = arn:aws:iam::~069:role/hc/hc-master
mfa_serial = arn:aws:iam::~183:mfa/username
external_id = ~069

AWS Credentials

[default]
aws_access_key_id = xxx
aws_secret_access_key = xxx


[GEHC-000]
aws_access_key_id = same as above
aws_secret_access_key = same as above

Policies assigned to IAM user

STS Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AssumeRole",
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": [
                "arn:aws:iam::*:role/hc/hc-master"
            ]
        }
    ]
}

User Policy

{
    "Statement": [
        {
            "Action": [
                "iam:*AccessKey*",
                "iam:*MFA*",
                "iam:*SigningCertificate*",
                "iam:UpdateLoginProfile*",
                "iam:RemoveUserFromGroup*"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:iam::~183:mfa/${aws:username}",
                "arn:aws:iam::~183:mfa/*/${aws:username}",
                "arn:aws:iam::~183:mfa/*/*/${aws:username}",
                "arn:aws:iam::~183:mfa/*/*/*${aws:username}",
                "arn:aws:iam::~183:user/${aws:username}",
                "arn:aws:iam::~183:user/*/${aws:username}",
                "arn:aws:iam::~183:user/*/*/${aws:username}",
                "arn:aws:iam::~183:user/*/*/*${aws:username}"
            ],
            "Sid": "Write"
        },
        {
            "Action": [
                "iam:*Get*",
                "iam:*List*"
            ],
            "Effect": "Allow",
            "Resource": [
                "*"
            ],
            "Sid": "Read"
        },
        {
            "Action": [
                "iam:CreateUser*",
                "iam:UpdateUser*",
                "iam:AddUserToGroup"
            ],
            "Effect": "Allow",
            "Resource": [
                "*"
            ],
            "Sid": "CreateUser"
        }
    ],
    "Version": "2012-10-17"
}

Force MFA Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BlockAnyAccessOtherThanAboveUnlessSignedInWithMFA",
            "Effect": "Deny",
            "NotAction": "iam:*",
            "Resource": "*",
            "Condition": {
                "BoolIfExists": {
                    "aws:MultiFactorAuthPresent": "false"
                }
            }
        }
    ]
}

main.tf

provider "aws" {
  profile                 = "GEHC-056"
  shared_credentials_file = "${pathexpand("~/.aws/config")}"
  region                  = "${var.region}"
}

data "aws_iam_policy_document" "test" {
  statement {
    sid    = "TestAssumeRole"
    effect = "Allow"

    actions = [
      "sts:AssumeRole",
    ]

    principals = {
      type = "AWS"

      identifiers = [
        "arn:aws:iam::~183:role/hc-devops",
      ]
    }

    sid    = "BuUserTrustDocument"
    effect = "Allow"

    principals = {
      type = "Federated"

      identifiers = [
        "arn:aws:iam::~875:saml-provider/ge-saml-for-aws",
      ]
    }

    condition {
      test     = "StringEquals"
      variable = "SAML:aud"
      values   = ["https://signin.aws.amazon.com/saml"]
    }
  }
}

resource "aws_iam_role" "test_role" {
  name               = "test_role"
  path               = "/"
  assume_role_policy = "${data.aws_iam_policy_document.test.json}"
}

Get Caller Identity

bash-4.4$ aws --profile GEHC-056 sts get-caller-identity
Enter MFA code for arn:aws:iam::772660252183:mfa/503072343:
{
  "UserId": "AROAIWCCLC2BGRPQMJC7U:botocore-session-1537474244",
  "Account": "730993910069",
  "Arn": "arn:aws:sts::730993910069:assumed-role/hc-master/botocore-session-1537474244"
}

And the error:

bash-4.4$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


Error: Error refreshing state: 1 error(s) occurred:

* provider.aws: Error creating AWS session: AssumeRoleTokenProviderNotSetError: assume role with MFA enabled, but AssumeRoleTokenProvider session option not set.

ydaetskcoR ydaetskcoR · Accepted Answer · 2018-09-21T08:15:26

Terraform doesn't currently support prompting for the MFA token when being ran as it is intended to be ran in a less interactive fashion as much as possible and it would apparently require significant rework of the provider structure to support this interactive provider configuration. There's more discussion about this in this issue.

As also mentioned in that issue the best bet is to use some form of script/tool that already assumes the role prior to running Terraform.

I personally use AWS-Vault and have written a small shim shell script that I symlink to from terraform (and other things such as aws that I want to use AWS-Vault to grab credentials for) that detects what it's being called as, finds the "real" binary using which -a, and then uses AWS-Vault's exec to run the target command with the specified credentials.

My script looks like this:

#!/bin/bash

set -eo pipefail

# Provides a shim to override target executables so that it is executed through aws-vault
# See https://github.com/99designs/aws-vault/blob/ae56f73f630601fc36f0d68c9df19ac53e987369/USAGE.md#overriding-the-aws-cli-to-use-aws-vault for more information about using it for the AWS CLI.

# Work out what we're shimming and then find the non shim version so we can execute that.
# which -a returns a sorted list of the order of binaries that are on the PATH so we want the second one.
INVOKED=$(basename $0)
TARGET=$(which -a ${INVOKED} | tail -n +2 | head -n 1)

if [ -z ${AWS_VAULT} ]; then
    AWS_PROFILE="${AWS_DEFAULT_PROFILE:-read-only}"
    (>&2 echo "Using temporary credentials from ${AWS_PROFILE} profile...")

    exec aws-vault exec "${AWS_PROFILE}" --assume-role-ttl=60m -- "${TARGET}" "$@"
else
    # If AWS_VAULT is already set then we want to just use the existing session instead of nesting them
    exec "${TARGET}" "$@"
fi

It will use a profile in your ~/.aws/config file that matches the AWS_DEFAULT_PROFILE environment variable you have set, defaulting to a read-only profile which may or may not be a useful default for you. This makes sure that AWS-Vault assumes the IAM role, grabs the credentials and sets them as environment variables for the target process.

This means that as far as Terraform is concerned it is being given credentials via environment variables and this just works.

Terraform unable to assume roles with MFA enabled

3 Answers

~/.aws/credentials

mfa.sh