Datadog integration inside docker containers

Question

New to datadog so I'm just really confused. First configuration was fast and simple. However as I want some app specific charts, it doesn't seem as clear as before for my current scenario.

We have one host with several docker machines, one for each service: - nginx - varnish - apache - database (mysql)

We've installed datadog client inside the host and also docker integration and everything works fine. What I don't get is how get metrics from apache or varnish, or whatever service that is inside docker. Reading the docs in varnish for example you have to execute:

$ sudo usermod -G varnish -a dd-agent

However, where should I run the command? dd-agent user exists only in the host, not in the docker container. Varnish is just the other way round.

Should I need to install the agent on each container?

It would be considered as another host for pricing?

In mysql case, I just have to configure the agent:

init_config:

instances:
  - server: localhost
    user: datadog
    pass: <UNIQUEPASSWORD>

    tags:
        - optional_tag1
        - optional_tag2
    options:

But as my host and the container are in separate routes, should I create a new docker container with the agent so it cat get to db container (changing server field)? Is it considered again as another host?

moebius moebius · Accepted Answer · 2018-08-21T03:48:18

In most cases, the datadog agent retrieves metrics from an integration by connecting to a URL endpoint. This would be the case for services such as nginx, mysql etc.

This means that you can run just one datadog agent on the host, and configure it to listen to URL endpoints of services exposed from each container.

For example, assuming a mysql docker container is run with the following command:

docker run -d \
  --name mysql \
  -p 3306:3306 \
  -e MYSQL_ROOT_PASSWORD=secret \
  -e MYSQL_DATABASE=mySchema \
  mysql

You can instruct the agent running on the host to connect to the container IP in the mysql.yaml agent configuration:

init_config:

instances:
- server: <container IP>
    user: datadog
    pass: secret

    tags:
        - optional_tag1
        - optional_tag2
    options:

Varnish is slightly different as the agent retrieves metrics using the varnishstat binary. According to the example template:

In order to support monitoring a Varnish instance which is running as a Docker container we need to wrap commands (varnishstat) with scripts which perform a docker exec on the running container.

To do this, on the host, create a wrapper script for the container:

echo "/usr/bin/docker exec varnish_container_name varnishstat "$@"" > /home/myuser/docker_varnish

Then specify the script location in the varnish.yaml agent configuration:

init_config:

instances:
    - varnishstat: /home/myuser/docker_varnish

Datadog integration inside docker containers

1 Answers