Trying to understand how describe_instances() work

Question

Trying to understand how describe_instances() work Here is a code which gives me instances ids of all the instances I have currently 3 so 3 instance id I get

    import boto3 
    from pprint import pprint

    ec2=boto3.client('ec2') 
    response=ec2.describe_instances()
    instancelist = [] 
   for reservation in (response["Reservations"]):
        for instance in reservation["Instances"]:
            instancelist.append(instance["InstanceId"]) print (instancelist)

gives me output

['i-03e7f6391a0f523ee', 'i-0e12c8dad5fb6d8c5', 'i-002adcd0913e4d673']

But if I write following

import boto3
from pprint import pprint
ec2=boto3.client('ec2')
response=ec2.describe_instances()
for x in response:
    print (x)
    print ("in for loop")

I just get following output

Reservations
in for loop
ResponseMetadata
in for loop

What I was expecting was instance id’s of instance. Now I wrote following

import boto3
from pprint import pprint

ec2=boto3.client('ec2')
response=ec2.describe_instances()

print (response["Reservations"][0]["Instances"][0]["InstanceId"])
print (response["Reservations"][1]["Instances"][0]["InstanceId"])
print (response["Reservations"][1]["Instances"][1]["InstanceId"])

I get output

i-03e7f6391a0f523ee i-0e12c8dad5fb6d8c5 i-002adcd0913e4d673

the above print statement does not work if written as below

print (response["Reservations"][2]["Instances"][0]["InstanceId"])

It gives error

    print (response["Reservations"][2]["Instances"][1]["InstanceId"])
IndexError: list index out of range

What I am not able to understand is how are the values being returned by describe_instances method in the statement

response=ec2.describe_instances()

Why the above for loop where I used x does not give output. I read the documentation here https://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Client.describe_instances and as far as I understand ec2.describe_instances() is returning a dictionary() Contains the output of DescribeInstances. • Reservations (list) -- Zero or more reservations. So with that understanding I thought I should be able to iterate over list members via

response["Reservations"][0]
response["Reservations"][1]
response["Reservations"][2]

But this is giving me error

print(response["Reservations"][2])
IndexError: list index out of range

I am unable to understand the response which is coming via ec2.describe_instances()

Output of print (response) as asked in below answer

    {'Reservations': [

    {'Groups': [], 'Instances': [{'AmiLaunchIndex': 0, 'ImageId': 'ami-d783a9b8', 

    'InstanceId': 'i-03e7f6391a0f523ee', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 

    'LaunchTime': datetime.datetime(2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 

    'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 

    'default'}, 'PrivateDnsName': 'ip-172-31-20-16.ap-south-1.compute.internal', 'PrivateIpAddress': 

    '172.31.20.16', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 

    'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 

    'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': 

    [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 1, 4, 28, 52, 

    tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-

    09716d3308f44c63f'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 

    'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 

    8, 1, 4, 28, 52, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-08d060230b617ca70', 

    'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '', 

    'Groups': [{'GroupName': 'launch-wizard-1', 'GroupId': 'sg-0e81c2a33e1039f58'}, {'GroupName': 

    'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:5a:17:52:69:a6', 

    'NetworkInterfaceId': 'eni-0146aab6d9503bf47', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-

    172-31-20-16.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.20.16', 

    'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-20-16.ap-south-

    1.compute.internal', 'PrivateIpAddress': '172.31.20.16'}], 'SourceDestCheck': True, 'Status': 

    'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': 

    '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'launch-wizard-1', 

    'GroupId': 'sg-0e81c2a33e1039f58'}, {'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 

    'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 

    'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 

    'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}], 'OwnerId': '11000101010', 'ReservationId': 

    'r-0571937a9ea83fac4'}, 

  {'Groups': [], 'Instances': [

{'AmiLaunchIndex': 0, 'ImageId': 'ami-00b6a8a2bd28daf19', 'InstanceId': 'i-0e12c8dad5fb6d8c5', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 'LaunchTime': datetime.datetime(2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 'default'}, 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-01d5ce67c9f1b081e'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-0cbdea888315049ae',  'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '',  'Groups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:f4:b8:bd:84:26', 'NetworkInterfaceId': 'eni-0d99a7669a1e4b9db', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39', 'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39'}], 'SourceDestCheck': True, 'Status': 'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}, 

{'AmiLaunchIndex': 1, 'ImageId': 'ami-00b6a8a2bd28daf19', 'InstanceId': 'i-002adcd0913e4d673', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 'LaunchTime': datetime.datetime (2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 'default'}, 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-0f0c49cc912a083f3'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-0f35d1842b76cff9a', 'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '', 'Groups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:21:d4:20:22:c6', 'NetworkInterfaceId': 'eni-0e3797492dc4e3299', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108', 'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108'}], 'SourceDestCheck': True, 'Status': 'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}], 'OwnerId': 'ReservationId': 'r-05cff59b2524ed79c'}], 

'ResponseMetadata': {'RequestId': 'fc80ae94-dd46-4c71-93f5-a38d6ede800c', 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-type':'text/xml;charset=UTF-8', 'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding', 'date': 'Sat, 25 Aug 2018 17:13:51 GMT', 'server': 'AmazonEC2'}, 'RetryAttempts': 0}}

You can note {'Groups': [], 'Instances': [{'AmiLaunchIndex': 0, line 2 times in above output and line 'AmiLaunchIndex': 1, 'ImageId': 'ami-00b6a8a2bd28daf19' two times while for the first instance it is 'AmiLaunchIndex': 0, 'ImageId': 'ami-d783a9b8', I had created these instances via run_instance method() as you can see in the print (response) output has come only but this is not giving a clear picture or I am unable to understand where are three instances in this this is what is not clear to me.

Answer 1

Basically, the response object will return one or many Reservation objects.

response=ec2.describe_instances()

You don't really have to worry about Reservations, other than for the purpose of iterating them. To delve into more detail, you could have a look at this past question,

https://serverfault.com/questions/749118/aws-ec2-what-is-a-reservation-id-exactly-and-what-does-it-represent

So your first block of code is an appropriate way of listing the instance ID's of all EC2 instances.

The second block of code is iterating through the keys in the response object, which is a dictionary,

response=ec2.describe_instances()
for x in response:
    print(x)

Here's the Boto documentation for the EC2 service, where you can find more info on the response object,

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_instances

Answer 2

As you can see the followinig error:

print(response["Reservations"][2])
IndexError: list index out of range

It states that you're trying to get the 2nd Instance of the Element Reservations whihch is not present in the given return. I would ask you to check the return of the function response=ec2.describe_instances() by printing it out on the terminal by using print(response) to get familiar with the data_type/return of that function. This will help you get a good grasp of what to expect the next time this function is called.

Also, when you know the kind of return that you're going to get, it will be easy for you to later iterate through the response using something like a for loop for lists like elements(which your function, response=ec2.describe_instances() returns according to it's documentation.) and then further use the response for whatever that you're trying to achieve.

Hope it helps. Happy Coding :)

Answer 3

You can also use list comprehension to get this data. Example:

  #!/usr/bin/env python3
  import boto3

  # Creating the boto client
  client = boto3.client('ec2')

  # Get the instance list
  instance_details = client.describe_instances(
    InstanceIds=[
      instance_id,
    ],
  )
  # Get all the Instance details from the Reservations
  # wrapping instances in [] prevents creating a outer list
  [instances] = [x['Instances']for x in i_details['Reservations']]