Logstash Input cloudwatch plugin not working

Question

Getting the following exception when trying to pull logs from aws cloudwatch. Please let me know how to get the plugin working.

logstash version :5.1.1 Cloudwatch Input plugin version :2.0.0

10:51:06.452 [[main]"AWS/Logs", metrics=>["IncomingLogEvents"], filters=>{"LogGroupName"=>"test"}, id=>"5d5114c61b58d374e42e44ca04d64215282cf5c1-1", enable_metric=>true, codec=>"plain_be95e7fb-ed07-4d9c-a765-1a1308e51254", enable_metric=>true, charset=>"UTF-8">, region=>"us-east-1", use_ssl=>true, statistics=>["SampleCount", "Average", "Minimum", "Maximum", "Sum"], interval=>900, period=>300, combined=>false> Error: No metrics to query Exception: RuntimeError

Aravind Aravind · Accepted Answer · 2018-01-23T05:40:53

Are you running this on your local machine or on AWS?

Local machine:

Make sure you set the AWS credentials in the logstash conf file like this or point to your local aws credentials YAML file using the aws_credentials property of the plugin

input {
    cloudwatch {
    namespace => "AWS/Logs"
    metrics => [ "IncomingBytes", "IncomingLogEvents"]
    filters => { "LogGroupName" => "/aws/lambda/your_lambda_name" }
    region => "us-east-1"
    access_key_id => "Insert access key ID"
    secret_access_key => "Insert secret access key"
    session_token => "Insert session token"
  }
}
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

Note: If you are using temporary AWS credentials make sure they are valid and have not expired.

EC2

Ensure that the IAM Instance Profile of the EC2 instance has access to cloud watch logs.

Reference - Logstash Cloudwatch Input plugin

Logstash Input cloudwatch plugin not working

1 Answers