Having CloudFormation wait for the user data

Question

I have a cloudformation stack which creates a EC2 instance and install something in it using UserData. Cloudformation immediately reports CREATE_COMPLETE upon creation of the EC2 instance based on RedHat. But at this point, the instance is not really usable since the userdata takes about 40 min to finish. I read through documentation and even tried cfn-signal but I could not successfully execute it.

Can someone tell me how exactly it has to be done?

EC2Instance:
  Type: AWS::EC2::Instance
  Properties:
    CreditSpecification:
      CPUCredits: standard
    IamInstanceProfile:
      Fn::ImportValue:
        !Sub ${InstanceProfileStackName}-instanceProfile
    ImageId: !Ref ImageId
    InstanceInitiatedShutdownBehavior: stop
    InstanceType: !Ref InstanceType
    SubnetId: !Ref SubnetId
    SecurityGroupIds:
      - !Ref DefaultSecurityGroup
      - !Ref WebSecurityGroup
    UserData:
      Fn::Base64: !Sub |
        #!/bin/bash
        set -e
        yum update -y

The above is the truncated part of my Cloudformation template.

UPDATE

I have the script which has the following line

source scl_source enable rh-python36

The default of my instance is python2.7 but I had to install my pip packages with python3.6. I am not sure if that was making the cfn-signal fail.

The script is going till the final step and seems to fail there. I am creating a recordset from the EC2 IP but Cloudformation still thinks the EC2 instance is not done and waiting till the timeout.

Screenshot of the instance snapshot

Log file end is as follows

Also my log file is named /var/log/cloud-init.log. There was no cloud-init-output.log in that directory.

Answer 1

You need two components:

• CreationPolicy so that CFN waits for a SUCCESS signal from the instance.

• cfn-signal helper script to perform the signalling action.

Thus your template could be modified as follows for Redhat 8:

EC2Instance:
  Type: AWS::EC2::Instance

  CreationPolicy: # <--- creation policy with timeout of 5 minutes
    ResourceSignal:
      Timeout: PT5M

  Properties:
    CreditSpecification:
      CPUCredits: standard
    IamInstanceProfile:
      Fn::ImportValue:
        !Sub ${InstanceProfileStackName}-instanceProfile
    ImageId: !Ref ImageId
    InstanceInitiatedShutdownBehavior: stop
    InstanceType: !Ref InstanceType
    SubnetId: !Ref SubnetId
    SecurityGroupIds:
      - !Ref DefaultSecurityGroup
      - !Ref WebSecurityGroup
    UserData:
      Fn::Base64: !Sub |
          #!/bin/bash -x

          yum update -y

          yum -y install python2-pip

          pip2 install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz

          python2 /usr/bin/cfn-signal -e $? \
                 --stack ${AWS::StackName} \
                 --resource EC2Instance \
                 --region ${AWS::Region}   

For debugging, as the user data may error out, have to login to the instance and check /var/log/cloud-init-output.log file

Answer 2

I could recreate your error and fixed here. Here is the corrected template. I added to the answer from Marcin

EC2Instance:
  Type: AWS::EC2::Instance
  CreationPolicy: 
    ResourceSignal:
      Timeout: PT5M # Specify the time here
  Properties:
    CreditSpecification:
      CPUCredits: standard
    IamInstanceProfile:
    Fn::ImportValue:
      !Sub ${InstanceProfileStackName}-instanceProfile
    ImageId: !Ref ImageId
    InstanceInitiatedShutdownBehavior: stop
    InstanceType: !Ref InstanceType
    SubnetId: !Ref SubnetId
    SecurityGroupIds:
      - !Ref DefaultSecurityGroup
      - !Ref WebSecurityGroup
    UserData:
      Fn::Base64: !Sub |
        #!/bin/bash -ex
        yum update -y
        source scl_source enable rh-python36
        <Your additional commands>
        cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}

You might want to counter check the indentation before trying.