How to dockerize the cloudwatch agent and collectd to collect the metrics?

Question

I need to collect the custom metrics from my ECS instances and from the documentation

These are steps I need to follow

Install aws cloudwatch agent
Install collectd daemon
Configure cloudwatch agent to get the metrics from collectd daemon

It seems I can

Dockerize the cloudwatch agent (which seems to be already done, but with lack of documentation)
Dockerize collectd daemon

Why to dockerize or not use awslogs driver for collecting metrics ?

Currently we already have some services running as docker instances managed by amazon ecs and configured to use awslog driver to send logs to amazon cloudwatch logs.

But in order to collect more custom metrics from the services e.g number of requests per particular user from service a the only solution which aws suggested is to use colletd with curl plugin along with cloud watch agent.

As due to some scaling issues, instead of running cloudwatch agent and collectd in a instance. I want to run those as containers.

Question:

Is there any way to run cloudwatch agent in docker container which can read the metrics from collectd daemon which runs in different container but on the same machine ?

Adiii Adiii · Accepted Answer · 2019-09-26T18:02:36

you do not need to run cloudwatch agent in your container, do not forget the rule of thumb, one process per container.

All you need to push application logs to stdout or stderr of the container and docker daemon will take care of it.

Important Configuration:

All you need to set log driver to AWS log driver in the task definition.

Amazon CloudWatch Logs logging driver

The awslogs logging driver sends container logs to Amazon CloudWatch Logs. Log entries can be retrieved through the AWS Management Console or the AWS SDKs and Command Line Tools.

Specifying a Log Configuration in your Task Definition

Before your containers can send logs to CloudWatch, you must specify the awslogs log driver for containers in your task definition. This section describes the log configuration for a container to use the awslogs log driver. For more information, see Creating a Task Definition.

   "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "awslogs-mysql",
                    "awslogs-region": "us-west-2",
                    "awslogs-stream-prefix": "awslogs-example"
                }
            }

AmazonECS-using_awslogs

How to dockerize the cloudwatch agent and collectd to collect the metrics?

1 Answers