Filtering out container logs from jorunald using fluentd

Question

I have around 30 docker containers which write the logs to journald with the tag 'container_name'. I'm using google-fluentd to send the logs to the Google Stackdriver. I'm finding it difficult to categorize the logs using the tag given in journald.

In google-fluentd.conf I tried tagging logs to Stackdriver one by one like shown below.

<source>
  @type systemd
  filters [{ "_SYSTEMD_UNIT": "test.service" }]
  <storage>
    @type local
    persistent true
    path /var/run/google-fluentd/test.service.pos
  </storage>
  read_from_head true
  tag test.service
</source>

This works, but if the number of services increases, I will have to include one block at a time. Without doing this, is there an easy way to get the container logs from journald from the tag and write to Stackdriver?

okkez okkez · Accepted Answer · 2019-04-09T00:30:54

The filters parameter is deprecated. Please use matches parameter instead.

https://github.com/reevoo/fluent-plugin-systemd/blob/master/docs/matching.md

Use @include directive to re-use your configuration blocks:

https://docs.fluentd.org/v1.0/articles/config-file#(6)-re-use-your-config:-the-%E2%80%9C@include%E2%80%9D-directive

Filtering out container logs from jorunald using fluentd

1 Answers