Docker filebeat autodiscover not detecting nginx logs

Question

On my mac I am running nginx in a docker file and filebeat in a docker file.

docker run -p 80:80 nginx

The above command successfully runs nginx which I can visit in the browser and the output is printed to the console.

This is my docker file that I am building and then running:

FROM docker.elastic.co/beats/filebeat:7.8.0
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root

And this is my filebeat.yml referenced in the above:

  filebeat.autodiscover:
    providers:
      - type: docker
        templates:
          - condition:
              contains:
                docker.container.image: nginx
            config:
              - type: docker
                containers.ids:
                    - "${data.docker.container.id}"
              - module: nginx
                access:
                  enabled: true
                  containers:
                      stream: "stdout"
                error:
                  enabled: true
                  containers:
                      stream: "stderr"
output.console:
    pretty: true

I run the filebeat docker image using:

 sudo docker run -it  -v /var/run/docker.sock:/var/run/docker.sock filebeat

The docker.sock stuff is because of an unable to connect to docker socket error I was getting (some kind of docker user error permissions I presume).

All of the above gets the filebeat running inside the docker container.

INFO    [autodiscover]  autodiscover/autodiscover.go:113    Starting autodiscover manager

However when I visit localhost in the browser and trigger the nginx log, the only output is occasional docker system metrics every 30 seconds:

{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":43}},"total":{"ticks":100,"time":{"ms":109},"value":100},"user":{"ticks":60,"time":{"ms":66}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"d3e79d62-6949-4d79-89e8-c595332c18ed","uptime":{"ms":30054}},"memstats":{"gc_next":10249440,"memory_alloc":5520104,"memory_total":17591608,"rss":55390208},"runtime":{"goroutines":23}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"console"},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":2},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}

The nginx log is not being detected.

This is a potentially relevant warning I am getting:

Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled

Edit: I think this is the crux of the issue, the harvester is not finding any logs:

filebeat":{"harvester":{"open_files":0,"running":0}}

Edit2: The root cause of the problem seems to be that on the Mac the terminal can't access the log files: /var/lib/docker/containers/${data.docker.container.id}/*.log as this aren't where they are stored on the Mac... so it seems running filebeat on mac is not supported this way between dockers

Fachtna Simi Fachtna Simi · Accepted Answer · 2020-07-03T23:33:17

Filebeat on Mac doesn't support collecting docker logs:

https://github.com/elastic/beats/issues/17310

Docker filebeat autodiscover not detecting nginx logs

1 Answers