CloudWatch agent doesn't recognize presence of IAM Role

Question

I'm trying to use the CloudWatch logs agent on a RedHat instance with an IAM role attached. The role has full access to CloudWatch. I installed and setup the agent using the instructions here:

http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html#running-ec2-step-2

Even though the IAM role is definitely attached to the instance, I keep seeing this message in /var/log/awslogs.log:

NoCredentialsError: Unable to locate credentials

When I run aws configure list, I can see the details for the IAM role.

  Name                    Value             Type           Location
  ----                    -----             ----           --------
 profile                <not set>           None           None
 access_key     ********************        iam-role
 secret_key     ********************        iam-role
 region                us-east-1            config-file    ~/.aws/config

Here is the contents of /var/awslogs/etc/aws.conf.

 [plugins]
 cwlogs = cwlogs
 [default]
 region = us-east-1

So why can't the CloudWatch logs agent find and use the IAM role?

When installing the agent, did you press ENTER when prompted for credentials? Press Enter if using an IAM role. Otherwise, enter your AWS access key ID. — helloV
Check /var/awslogs/etc/aws.conf. If it has entries for aws_access_key_id and aws_secret_access_key, remove them and then restart the agent. — helloV
I pressed Enter when it asked for the access and secret key. /var/awslogs/etc/aws.conf doesn't contain aws_access_key_id or aws_secret_access_key. Edited the OP for clarity. — jcgrowley

jcgrowley jcgrowley · Accepted Answer · 2017-09-05T20:42:15

So after much banging my head against the wall, I finally figured out what my problem was. I'm using a proxy to enable the CloudWatch agent to communicate with CloudWatch, and I forgot to add NO_PROXY=169.254.169.254 to /var/awslogs/etc/proxy.conf. So when the agent attempted to query the metadata for information about the IAM role, it tried to go through the proxy to get it. Once I added the NO_PROXY in, it worked fine.

CloudWatch agent doesn't recognize presence of IAM Role

1 Answers