Write journald metadata to rsyslog

2
votes

I have a setup where docker containers use the journald log driver to write their logs. Currently log lines from the journal are forwarded to rsyslog running on the host, but the application name on the syslog lines appears as dockerd.

As a workaround, I'd like to write the CONTAINER_NAME field form the journal metadata into the line that appears in syslog, so I can identify what container wrote what line after the host's syslog has been shipped to a syslog aggregation server.

Any suggestions?

3
loggingdockersystemdrsyslog

3 Answers

1
votes

I was able to achieve this by defining a template after parsing the structured logs from journald. For figuring out what properties were available I ran journalctl -o verbose -n 10

rsyslog has multiple different ways to do the same configuration, here is my config from a CentOS 7 machine:

module(load="imjournal" StateFile="imjournal.state") # Load imjournal module
module(load="mmjsonparse") # Load mmjsonparse module for structured logs

action(type="mmjsonparse") # Attempt to parse JSON

template(name="ContainerTemplate" type="list") {
    property(name="timestamp" dateFormat="rfc3339")
    constant(value=" ")
    property(name="$!CONTAINER_NAME")
    constant(value=" ")
    property(name="$!CONTAINER_ID")
    constant(value=" ")
    property(name="$!MESSAGE")
    constant(value="\n") # Separate logs with a newline
}

if ($!CONTAINER_NAME != "") then {
    action(type="omfile" file="/var/log/messages" template="ContainerFormat")
} else {
    *.info;mail.none;news.none;authpriv.none;cron.none action(type="omfile" file="/var/log/messages")
}

Reference documentation:

1
votes

I think closest you could get image name. You can add a log tag to show the image name in logs. This feature has been added in v1.11.0. For example:

docker run --log-driver=journald --log-opt tag="{{.ImageName}}

Have a look at log tag docs too. Hope this helps.

0
votes

I managed to get this working by putting this in rsyslog.conf

if ( $!CONTAINER_TAG == "mycontainer" ) then { action(type="omfile" file="/var/log/mycontainer.log") stop } I can test that it works with this docker run --log-driver=journald --log-opt tag="mycontainer" centos:latest echo Pierre7 I get my log both in the file /var/log/mycontainer and in the journal, which I can find with

journalctl --unit docker CONTAINER_TAG=mycontainer It is difficult to find the information, the only place where it seems documented in in the redhat documentation: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/s1-structured_logging_with_rsyslog.html

My use case is that I want the journal for all the benefits it gives but the support people are requesting to have normal files. Also we will use the splunkforwarder to consume the files.