cnf-init throws Python error on ubuntu ami

Question

I'm using the below aws cloudformation template to spin up an autoscaling group and it's throwing a python error when I try to run cfn-init. I've tried everything I can think of to resolve this but I'm now totally stumped. The error I get is:

Error occurred during build: list indices must be integers, not unicode

This is my Cloudformation template:

Description: 'Autoscaling groups'

Parameters:
  DeploymentProfileARN: 
    Description: The ARN of the iam group with deployment permissions.
    Type: String
  PublicSecurityGroup: 
    Description: The security group for use with ec2.
    Type: String
  TargetGroup:
    Description: The load balancer target group
    Type: String
  Subnets: 
    Description: The load balancer target group
    Type: 'List<AWS::EC2::Subnet::Id>'
  DomainUrl:
    Description: The url of the site excluding any sub domains eg. "example.com"
    Type: String
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String
    Default: Dev
    AllowedValues: [Prod, Uat, Dev]
  MasterDataBaseAddress:
    Description: The address of the master database
    Type: String

Resources:
  AuthScalingLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Metadata: 
      AWS::CloudFormation::Init:
        nginx:
          packages:
            apt: 
              - "nginx-core"
          files:
            '/tmp/test':
              content: !Sub |
                test
            '/etc/nginx/sites-enabled/default':
              content: !Sub |
                  upstream phoenix {
                    server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
                  }
                  server {
                    listen 80 default_server;
                    server_name .${DomainUrl};
                    location / {
                      allow all;
                      proxy_http_version 1.1;
                      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                      proxy_set_header Host $http_host;
                      proxy_set_header X-Cluster-Client-Ip $remote_addr;
                      proxy_set_header Upgrade $http_upgrade;
                      proxy_set_header Connection "upgrade";
                      proxy_pass http://phoenix;
                    }
                  }
          commands:
            01-restartNginx:
              command: service nginx restart
        configSets:
          setup:
            - "nginx"
    Properties:
      ImageId: "ami-0701e7be9b2a77600"
      InstanceType: t1.micro
      KeyName: "main"
      IamInstanceProfile: !Ref DeploymentProfileARN
      SecurityGroups:
        - !Ref PublicSecurityGroup
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash

          # install cfn
          apt update
          apt install -y python-pip 
          mkdir -p /opt/aws/bin
          pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gpip

          cfn-init --stack ${AWS::StackName} --resource AuthScalingLaunchConfig --region ${AWS::Region} --configsets setup
          service cfn-hup start
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties: 
      AvailabilityZones: 
        - !Select [ 0, !GetAZs "" ]
        - !Select [ 1, !GetAZs "" ]
      LaunchConfigurationName: !Ref AuthScalingLaunchConfig
      TargetGroupARNs:
        - !Ref TargetGroup
      HealthCheckGracePeriod: "60"
      HealthCheckType: EC2
      VPCZoneIdentifier: !Ref Subnets
      MaxSize: "10"
      DesiredCapacity: "1"
      MinSize: "1"

Any help or steers in the right direction would be awesome. I've tried using different versions of the aws-cfn-bootstrap but they've all given me the same issue.

Answer 1

I've managed to get this working. For the future generations this worked:

Description: 'Autoscaling groups'

Parameters:
  DeploymentProfileARN: 
    Description: The ARN of the iam group with deployment permissions.
    Type: String
  PublicSecurityGroup: 
    Description: The security group for use with ec2.
    Type: String
  TargetGroup:
    Description: The load balancer target group
    Type: String
  Subnets: 
    Description: The load balancer target group
    Type: 'List<AWS::EC2::Subnet::Id>'
  DomainUrl:
    Description: The url of the site excluding any sub domains eg. "example.com"
    Type: String
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String
    Default: Dev
    AllowedValues: [Prod, Uat, Dev]
  MasterDataBaseAddress:
    Description: The address of the master database
    Type: String

Resources:
  AuthScalingLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Metadata: 
      AWS::CloudFormation::Init:
        nginx:
          packages:
            apt:
              'nginx-core': []
          files:
            '/etc/nginx/sites-enabled/default':
              content: !Sub |
                  upstream phoenix {
                    server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
                  }
                  server {
                    listen 80 default_server;
                    server_name .${DomainUrl};
                    location / {
                      allow all;
                      proxy_http_version 1.1;
                      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                      proxy_set_header Host $http_host;
                      proxy_set_header X-Cluster-Client-Ip $remote_addr;
                      proxy_set_header Upgrade $http_upgrade;
                      proxy_set_header Connection "upgrade";
                      proxy_pass http://phoenix;
                    }
                  }
          commands:
            01-restartNginx:
              command: service nginx restart
        configSets:
          setup:
            - "nginx"
    Properties:
      ImageId: "ami-0701e7be9b2a77600"
      InstanceType: t1.micro
      KeyName: "main"
      IamInstanceProfile: !Ref DeploymentProfileARN
      SecurityGroups:
        - !Ref PublicSecurityGroup
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash

          # install cfn
          apt-get -y update
          apt-get -y install python-pip
          pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
          cp /usr/local/init/ubuntu/cfn-hup /etc/init.d/cfn-hup 
          chmod +x /etc/init.d/cfn-hup 
          update-rc.d cfn-hup defaults
          service cfn-hup start

          # install code deploy
          cd /home/ubuntu
          wget https://aws-codedeploy-eu-west-1.s3.amazonaws.com/latest/install
          chmod +x ./install
          ./install auto

          cfn-init --stack ${AWS::StackName} --resource AuthScalingLaunchConfig --region ${AWS::Region} --configsets setup
  RibbonAutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties: 
      AvailabilityZones: 
        - !Select [ 0, !GetAZs "" ]
        - !Select [ 1, !GetAZs "" ]
      LaunchConfigurationName: !Ref AuthScalingLaunchConfig
      TargetGroupARNs:
        - !Ref TargetGroup
      HealthCheckGracePeriod: "60"
      HealthCheckType: EC2
      VPCZoneIdentifier: !Ref Subnets
      MaxSize: "10"
      DesiredCapacity: "1"
      MinSize: "1"

Answer 2

I have made the following change for UserData which will get latest cfn-init.

Description: 'Autoscaling groups'

Parameters:
  DeploymentProfileARN: 
    Description: The ARN of the iam group with deployment permissions.
    Type: String
  PublicSecurityGroup: 
    Description: The security group for use with ec2.
    Type: String
  TargetGroup:
    Description: The load balancer target group
    Type: String
  Subnets: 
    Description: The load balancer target group
    Type: 'List<AWS::EC2::Subnet::Id>'
  DomainUrl:
    Description: The url of the site excluding any sub domains eg. "example.com"
    Type: String
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names
    Type: String
    Default: Dev
    AllowedValues: [Prod, Uat, Dev]
  MasterDataBaseAddress:
    Description: The address of the master database
    Type: String

Resources:
  AuthScalingLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Metadata: 
      AWS::CloudFormation::Init:
        nginx:
          packages:
            apt: 
              - "nginx-core"
          files:
            '/tmp/test':
              content: !Sub |
                test
            '/etc/nginx/sites-enabled/default':
              content: !Sub |
                  upstream phoenix {
                    server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
                  }
                  server {
                    listen 80 default_server;
                    server_name .${DomainUrl};
                    location / {
                      allow all;
                      proxy_http_version 1.1;
                      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                      proxy_set_header Host $http_host;
                      proxy_set_header X-Cluster-Client-Ip $remote_addr;
                      proxy_set_header Upgrade $http_upgrade;
                      proxy_set_header Connection "upgrade";
                      proxy_pass http://phoenix;
                    }
                  }
          commands:
            01-restartNginx:
              command: service nginx restart
        configSets:
          setup:
            - "nginx"
    Properties:
      ImageId: "ami-0701e7be9b2a77600"
      InstanceType: t1.micro
      KeyName: "main"
      IamInstanceProfile: !Ref DeploymentProfileARN
      SecurityGroups:
        - !Ref PublicSecurityGroup
      UserData:
        Fn::Base64:
          !Sub
            - |
                #!/bin/bash
                # install cfn
                apt-get update -y
                apt-get install -y python-pip curl gzip python3-pip
                pip install awscli --upgrade
                pip install --upgrade pip
                python3 -m pip install --upgrade pip
                python3 -m pip install --upgrade awscli
                cd ~
                if [ ! -d ~/aws-cfn-bootstrap-latest ]; then
                  apt-get install -y python-setuptools jq
                  # Get the latest CloudFormation package
                  cd ~
                  mkdir aws-cfn-bootstrap-latest
                  curl https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz | tar xz -C aws-cfn-bootstrap-latest --strip-components 1
                  easy_install aws-cfn-bootstrap-latest
                fi
                # Run CFN-Init
                echo "cfn-init starting" 2>&1;
                /usr/local/bin/cfn-init -s ${AWS::StackId} -r AuthScalingLaunchConfig --region ${AWS::Region} -c setup
                echo "cfn-init complete" 2>&1;
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties: 
      AvailabilityZones: 
        - !Select [ 0, !GetAZs "" ]
        - !Select [ 1, !GetAZs "" ]
      LaunchConfigurationName: !Ref AuthScalingLaunchConfig
      TargetGroupARNs:
        - !Ref TargetGroup
      HealthCheckGracePeriod: "60"
      HealthCheckType: EC2
      VPCZoneIdentifier: !Ref Subnets
      MaxSize: "10"
      DesiredCapacity: "1"
      MinSize: "1"