K8S EFK (especially Fluentd) daemonset for docker journald logging driver

2
votes

Question

Are there known available Fluentd daemonset for journald docker logging driver so that I can send K8S pod logs to Elasticsearch?

Background

As in add support to log in kubeadm, the default logging driver for K8S installed by kubeadm is journald.

the community is collectively moving away from files on disk at every place possible in general, and this would unfortunately be a step backwards. ...

You can edit you /etc/docker/daemon.json to set its default log to json files and set a max size and max files to take care of the log rotation. After that, logs wont be written to journald and you will be able to send your log files to ES.

However, the K8S EFK addon and Fluentd K8S or Aggregated Logging in Tectonic still expect to look for files in /var/log/containers in the host, if I understood correctly.

It looks Alternative fluentd docker image designed as a drop-in replacement for the fluentd-es-image looks to be adopting journald driver. However could not make it run the pods.

1
kubernetesfluentdefk

1 Answers

1
votes

docker log driver journald send docker logs to systemd-journald.service

so, we need to make systemd-journald persistent save to /var/log/journal

edit /etc/systemd/journald.conf:

...
[Journal]
Storage=persistent
#Compress=yes
...

then restart to apply changes:

systemctl restart systemd-journald
ls -l /var/log/journal

as /var/log has been mounted into fluentd pod, its all done, restart fluentd pod it works for me @202104.

by the way, i am using fluentd yaml from:

and the env FLUENTD_SYSTEMD_CONF value should not be disable