Logstash aggregate docker logs (to fix multiline issues)

0
votes

I have logentries coming into logstash from different containers. Each logentry contains a field ContainerID and a field Message. Any messages with the same ContainerID I assume are ordered properly.

I want to merge the contents of messages of multiple logentries with the same containerId like the following:

  1. PREFIX | String1
  2. String2
  3. String3
  4. PREFIX | String4

So that I end up with logentries:

  1. PREFIX | String1 \n String2 \n String 3
  2. PREFIX | String4

Can this be done with the logstash aggregate filter plugin?

Specifically, will there be issues if messages with a different containerID are received in between 1., 2., 3., and 4. from above?

Background:

I have multiple docker containers generating log entries which are sent to logstash using the docker gelf log driver from multiple docker deamons (in a swarm).

All of this works and logstash receives and filters the logentries.

Docker views each line as a separate logentry and forwards it to logstash. In case of multiline logentries this is a problem. I want to "merge" many logentries which correspond to one multiline message into one. I know when a "logical" logentry ends because the next entry will start with PREFIX which is why I suggest the solution above.

1
dockerlogstashdocker-swarmlogstash-configurationgelf

1 Answers

1
votes

I suggest to use Filebeat for that. It has multiline support and with the latest release you can make use of the Autodiscover functionality for Docker.

I recommend to read the discussion in: https://github.com/elastic/beats/issues/918