Unable to create EC2 when subnet is used via Ansible(same works through AWS-CLI)

Question

I am trying to create EC2 instance using ansible. If I try this without the subnet(and default security group), it works perfect and creates EC2. But this is not I want. I want to create instance using a specific 'sg' and using the subnet that's already existing(defined by my organization).

Same subnet and 'sg' works fine when using AWS-CLI(and via console too), same profile, same image, same key and same instance type. It creates instance under my subnet and assigns the sg passed in the command - Perfect!! Can we rule out the access/role related issues here(as CLI/console works fine)? If so, what else issue can be with Ansible/boto?

AWS CLI:

aws ec2 run-instances --image-id ami-3d401234 --count 1 --instance-type t2.large --region us-east-1 --key-name MyKeyNameHere --security-group-ids sg-766b1234 --subnet-id subnet-09871234 --profile MyProfileNameHere

Here is the playbook.

 - name: Provision an EC2 node
    hosts: local
    connection: local
    gather_facts: false
    tags: provisioning
    vars:
      instance_type: t2.large
      image: ami-3d401234
      group_id: sg-766b1234
      region: us-east-1
      keypair: MyKeyNameHere
      vpc_subnet_id: subnet-09871234

    tasks:
      - name: Launch new Instance
        local_action: ec2 instance_tags="Name=MyInstance"
                      instance_type={{ instance_type}}
                      image={{ image }}
                      wait=true
                      group_id={{ group_id }}
                      profile=MyProfileNameHere
                      region={{ region }}
                      vpc_subnet_id={{ vpc_subnet_id }}
                      keypair={{ keypair }}
        register: ec2

And here is the error, not sure why 401 again(got this earlier when profile was not mentioned in the playbook). I am sure access and secret keys are correct because i am able to create with default sg.

    vpc_id = vpc.get_all_subnets(subnet_ids=[vpc_subnet_id])[0].vpc_id
  File "/Library/Python/2.7/site-packages/boto-2.38.0-py2.7.egg/boto/vpc/__init__.py", line 1153, in get_all_subnets
    return self.get_list('DescribeSubnets', params, [('item', Subnet)])
  File "/Library/Python/2.7/site-packages/boto-2.38.0-py2.7.egg/boto/connection.py", line 1186, in get_list
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 401 Unauthorized
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>6182f17d-f62e-4d57-b351-3498dc8a53b7</RequestID></Response>

And i have the access key and secret key information in ~/.boto file. Just the aws_access_key_id and aws_secret_access_key. No IAM role information, etc.

It is possible the credentials that you use in ~/.boto is not the same as the one in ~/.aws/credentials — helloV
They are the same, double checked it now - again. I don't get '401' once i remove the subnet from playbook. — Raghu
Okay, checked the differences between my ~/.aws/credentials file and .boto. This was missing in boto, aws_security_token. Never thought this would be needed as i was already passing the access and secret keys. I guess it is needed a i am given access part of an organization group? I added this and it works now. Thanks @helloV for making me check the differences again with you comment :-) — Raghu

Raghu Raghu · Accepted Answer · 2015-10-27T17:43:33

This was missing in boto, aws_security_token. Never thought this would be needed as i was already passing the access and secret keys. I guess it is needed as i am given access part of an organization group? I added this and it works now. Thanks @helloV for making me check the differences again with you comment :-)

Unable to create EC2 when subnet is used via Ansible(same works through AWS-CLI)

1 Answers