How to copy a file and add dynamic content on startup of an EC2 instance using CloudFormation?

Question

I need to copy the content of a cert file which comes from the secretsmanager into an EC2 instance on startup using CloudFormation.

Edit:

I added an IAM Role, a Policy, and an InstanceProfile in my code to ensure that I can access the SecretsManager value using UserData

The code looks like this now:

SecretsManagerAccessRole:
Type: AWS::IAM::Role
Properties:
  RoleName: CloudFormationSecretsManagerAccessRole
  AssumeRolePolicyDocument:
    Version: '2012-10-17'
    Statement:
      - Effect: Allow
        Principal:
          AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
        Action: sts:AssumeRole
  Path: "/"

SecretsManagerInstanceProfile:
  Type: AWS::IAM::InstanceProfile
  Properties:
    Path: "/"
    Roles: [ !Ref SecretsManagerAccessRole ]

SecretsManagerInstancePolicy:
  Type: AWS::IAM::Policy
  Properties:
    PolicyName: SecretsManagerAccessPolicy,
    PolicyDocument:
      Statement:
        - Effect: Allow
          Action: secretsmanager:GetSecretValue
          Resource: <arn-of-the-secret>
  Roles: [ !Ref SecretsManagerAccessRole ]

LinuxEC2Instance:
  Type: AWS::EC2::Instance
  Properties:
    IamInstanceProfile: !Ref SecretsManagerInstanceProfile
    UserData:
      Fn::Base64: !Sub |
        #!/bin/bash -xe
        yum update -y

        groupadd -g 110 ansible
        adduser ansible -g ansible

        mkdir -p /home/ansible/.ssh
        chmod 700 /home/ansible/.ssh

        aws  secretsmanager  get-secret-value \
          --secret-id <arn-of-the-secret> \
          --region ${AWS::Region} \
          --query 'SecretString' \
          --output text > /home/ansible/.ssh/authorized_keys

        chmod 000644 /home/ansible/.ssh/authorized_keys
        chown -R ansible.ansible /home/ansible/.ssh/

        cat /home/ansible/.ssh/authorized_keys

During startup of the instance, I get this issue here:

Unable to locate credentials. You can configure credentials by running "aws configure".

It seems like the user did not get the necessary role to perform this action in UserData? Why is that?

Answer 1

I tried few things, but all failed. The only thing that worked was to use UserData.

For example, you could have the following:

  LinuxEC2Instance:
    Type: AWS::EC2::Instance
    
    Properties:                
      ImageId: ami-08f3d892de259504d # AL2 in us-east-1
      InstanceType: t2.micro
      IamInstanceProfile: <name-of-instance-profile>
      KeyName: MyKeyPair     
      UserData: 
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          yum update -y
          
          groupadd -g 110 ansible
          adduser ansible -g ansible
          
          mkdir -p /home/ansible/.ssh
          chmod 700 /home/ansible/.ssh
          
          secret_value=$(aws secretsmanager  get-secret-value \
              --secret-id <arn-of-the-secret> \
              --region ${AWS::Region} \
              --query 'SecretString' \
              --output text)

          # have to check the exact command here of jq
          echo ${!secret_value} | jq -r '.key' > /home/ansible/.ssh/authorized_keys              
              
          chmod 000644 /home/ansible/.ssh/authorized_keys
          chown -R ansible.ansible /home/ansible/.ssh/

You also would need to add an instance role/profile to the instance so that it can read the secret. The role could contain the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ReadSecretValue",
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": "<arn-of-secret>"
        }
    ]
}

edit:

In you KMS is used for encryption of the secret, the instance role would need to have permissions for KMS as well.

Answer 2

Ok, I got it working, this is the full answer, the code below worked for me, in addition, I needed to add 'kms:GenerateDataKey', 'kms:Decrypt' to the permissions for it to properly retrieve the secret, finally I needed to use jq to retrieve the value out of the JSON format I got from secrets manager:

  CFNInstanceProfile:
  Type: AWS::IAM::InstanceProfile
  Properties:
    Path: /
    Roles:
      - !Ref CFNAccessRole
  CFNAccessRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: CFNAccessRole
    AssumeRolePolicyDocument:
      Version: 2012-10-17
      Statement:
        - Effect: Allow
          Principal:
            Service: ec2.amazonaws.com
          Action: sts:AssumeRole
    Path: /
  CFNInstancePolicy:
  Type: AWS::IAM::Policy
  Properties:
    PolicyName: SecretsManagerAccessPolicy,
    PolicyDocument:
      Statement:
        - Effect: Allow
          Action: ['secretsmanager:GetSecretValue', 'kms:GenerateDataKey', 'kms:Decrypt']
          Resource: '*'
    Roles:
      - !Ref CFNAccessRole

# EC2 Instance creation
LinuxEC2Instance:
  Type: AWS::EC2::Instance
  Properties:
    IamInstanceProfile: !Ref CFNInstanceProfile
    UserData:
      Fn::Base64: !Sub |
        #!/bin/bash -xe
        yum update -y

        groupadd -g 110 ansible
        adduser ansible -g ansible

        mkdir -p /home/ansible/.ssh
        chmod 700 /home/ansible/.ssh

        aws secretsmanager get-secret-value \
            --secret-id <arn-of-the-secret>  \
          --region ${AWS::Region} \
          --query 'SecretString' \
          --output text | jq -r ".key" > /home/ansible/.ssh/authorized_keys

        chmod 000644 /home/ansible/.ssh/authorized_keys
        chown -R ansible.ansible /home/ansible/.ssh/

        cat /home/ansible/.ssh/authorized_keys