How to install aws-cfn-bootstrap/cfn-init package in Redhat using CloudFormation?

Question

I am trying to launch a instance with CloudFormation Template. Instance was started but the UserData section was not executed completely because cfn-init/aws-cfn-bootstrap package was not installed in Redhat 7 AMI. I tried installing aws-cfn-bootstrap package manually but could not install due to the conflicts with python version.

Here is the UserData section of CloudFormation Template

"UserData": {
                "Fn::Base64": {
                    "Fn::Join": [
                        "\n",
                        [
                            "#!/bin/bash",
                            "set -x",
                            "",
                            "INSTANCE_ID=`/opt/aws/bin/ec2-metadata --instance-id | cut -f2 -d' '`",
                            "REGION=`/opt/aws/bin/ec2-metadata --availability-zone| cut -f2 -d' ' | sed '$s/.$//'`",
                            {
                                "Fn::Join": [
                                    "",
                                    [
                                        "AID='",
                                        {
                                            "Fn::GetAtt": [
                                                "eip",
                                                "AllocationId"
                                            ]
                                        },
                                        "'"
                                    ]
                                ]
                            },
                            "aws ec2 associate-address --region $REGION --instance-id $INSTANCE_ID --allocation-id $AID"
                        ]
                    ]
                }
            }

cloud-init.log

Nov 12 03:55:27 localhost cloud-init: Cloud-init v. 0.7.6 running 'modules:config' at Thu, 12 Nov 2015 08:55:27 +0000. Up 19.01 seconds. 
Nov 12 03:55:28 localhost cloud-init: Cloud-init v. 0.7.6 running 'modules:final' at Thu, 12 Nov 2015 08:55:27 +0000. Up 19.67 seconds. 
Nov 12 03:55:28 localhost cloud-init: ++ /opt/aws/bin/ec2-metadata --instance-id 
Nov 12 03:55:28 localhost cloud-init: /var/lib/cloud/instance/scripts/part-001: line 4: /opt/aws/bin/ec2-metadata: No such file or directory 
Nov 12 03:55:28 localhost cloud-init: ++ cut -f2 '-d ' 
Nov 12 03:55:28 localhost cloud-init: + INSTANCE_ID= 
Nov 12 03:55:28 localhost cloud-init: ++ cut -f2 '-d ' 
Nov 12 03:55:28 localhost cloud-init: ++ sed '$s/.$//' 
Nov 12 03:55:28 localhost cloud-init: ++ /opt/aws/bin/ec2-metadata --availability-zone 
Nov 12 03:55:28 localhost cloud-init: /var/lib/cloud/instance/scripts/part-001: line 5: /opt/aws/bin/ec2-metadata: No such file or directory 
Nov 12 03:55:28 localhost cloud-init: + REGION= 
Nov 12 03:55:28 localhost cloud-init: + AID=eipalloc-XXXXXX 
Nov 12 03:55:28 localhost cloud-init: + aws ec2 associate-address --region --instance-id --allocation-id eipalloc-XXXXXX 
Nov 12 03:55:28 localhost cloud-init: /var/lib/cloud/instance/scripts/part-001: line 7: aws: command not found 
Nov 12 03:55:28 localhost cloud-init: 2015-11-12 03:55:28,078 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [127] 
Nov 12 03:55:28 localhost cloud-init: 2015-11-12 03:55:28,089 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts) 
Nov 12 03:55:28 localhost cloud-init: 2015-11-12 03:55:28,089 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/pyt
hon2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed 

Answer 1

Here's what worked for me, i'm not taking full credit for this as I found it on one of the aws forums but now i can't find the source link...anyway i had to modify it a bit to work for me on redhat 6 so hopefully it helps someone else as well.

This is the Userdata section in the template to install the cfn-init scripts:

"UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
      "#!/bin/bash -xe\n",

      "### This is redhat 6. It supports cloud-init but Cfn packages need to be installed unlike AWS Linux. And they are installed in a different location\n",
      "# First enable EPEL\n",
      "rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm", "\n",
      "# Now install Python Setuptools(easy_install) and Pip", "\n",
      "yum -y install python-pip", "\n",
      "# Now install cfn scripts", "\n",
      "/usr/bin/easy_install --script-dir /opt/aws/bin https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz", "\n",
      "# Now fix the cfn-hup script and copy to init.d location as AWS does not do it for you", "\n",
      "cp -f `pip show aws-cfn-bootstrap 2> /dev/null | egrep  \"^Location\" | awk -F \":\" ' { print $2 }'`/init/redhat/cfn-hup /etc/init.d/", "\n",
      "chmod 755 /etc/init.d/cfn-hup", "\n",
      "chkconfig --add cfn-hup", "\n",

      "/opt/aws/bin/cfn-init -v ",
      " --stack ", { "Ref" : "AWS::StackId" },
      " --resource AppServer ",
      " --configsets Install ",
      " --region ", { "Ref" : "AWS::Region" }, "\n"
    ]]}}

Answer 2

You can install it in this way by using only pip (no need to use easy_install):

Python 2:

pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz

Python 3:

pip3 install  https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz

Full example with CloudFormation LaunchConfiguration:

LaunchConfig:
  Type: AWS::AutoScaling::LaunchConfiguration
  Properties:
    UserData:
      Fn::Base64:
        !Sub |
          #!/bin/bash

          set -e # Exit on error
          set -u # Treat unset variable as error
          set -x # Trace commands
          set -o pipefail

          # Install latest security updates
          yum update -y

          # Install pip
          yum -y install python3-pip

          # Install AWS CLI and CFN Bootstrap
          pip3 install awscli
          pip3 install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz

Answer 3

For RHEL-7.x, you have to add the pypa-repo, below is something worked for me.

  "UserData": {
      "Fn::Base64": {
        "Fn::Join": [
          "",
          [
            "#!/bin/bash\n",
            "## Install EPEL\n",
            "rpm -Uvh http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-9.noarch.rpm\n",
            "# Installing CFN-init \n",
            "yum -y install python-pip",
            "\n",
            "cd /use/bin", "\n",
            "/usr/bin/easy_install --script-dir /opt/aws/bin https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz",
            "\n",
            "cp -f `pip show aws-cfn-bootstrap 2> /dev/null | grep -i \"^Location\" | awk   '{ print $2\"/init/redhat/cfn-hup\"}'` /etc/init.d/",
            "\n",
            "chmod 755 /etc/init.d/cfn-hup",
            "\n",
            "chkconfig --add cfn-hup",
            "\n",
            "## DONE \n",
            "# Install the files and packages from the metadata\n",
            "/opt/aws/bin/cfn-init -v ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource appInstance1 ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n",
            "# Start up the cfn-hup daemon to listen for changes to the Web Server metadata\n",
            "/opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup'\n",
            "# Signal the status from cfn-init\n",
            "/opt/aws/bin/cfn-signal -e $? ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource appInstance1 ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n"
          ]
        ]
      }
    }

Answer 4

Example for RedHat 8:

  UserData: 
    Fn::Base64: !Sub |
          #!/bin/bash -x

          yum update -y

          yum -y install python2-pip

          pip2 install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz

          python2 /usr/bin/cfn-signal -e $? \
                 --stack ${AWS::StackName} \
                 --resource MyInstance \
                 --region ${AWS::Region}  

Where MyInstance is the name of the instance resource in your CFN.