Rsyslog parsing issue - no header in syslog message that is sent to Rsyslog

Question

I have an applience which sends logs to an Rsyslog server over TCP as JSON Stream. however it doesn't put any header in the syslog message. The sample raw message that is sent is below:

{"query_class":"C_INTERNET","source_ip":"10.1.1.1","query_type":"A","trans_id":502,"err_code":"NXDOMAIN","@type":"dns","dest_ip":"10.10.10.1","dest_port":53,"uid":"CDdCwH1lD2ToFS5y02","epochdate":1595000476.3491,"query":"www.google.com","@host":"host-10-01","rejected":true,"source_port":54764,"proto":"udp"}

Rsyslog forwards this message to another server and puts a header in the format like below(there is no template applied):

<13>Jun 16 10:43:09 host01.example.local

I don't know where this header comes from(maybe a default config is applied when there is no header in the message).

The problem is, the TAG is missing in the header and it causes the raw message be parsed improperly.

When Rsyslog forwards the logs to another machine or writes it to a file, this is how it looks:

<13>Jun 16 10:43:09 host01.example.local {"query_class" "C_INTERNET","source_ip":"10.1.1.1","query_type":"A","trans_id":502,"err_code":"NXDOMAIN","@type":"dns","dest_ip":"10.10.10.1","dest_port":53,"uid":"CDdCwH1lD2ToFS5y02","epochdate":1595000476.3491,"query":"www.google.com","@host":"host-10-01","rejected":true,"source_port":54764,"proto":"udp"}

{"query_class" gets extraced and becomes the TAG (":" is removed). The rest becomes the syslog message.

Unfortunately, it is not possible to make any change at the source to configure a header. How can I add a header to this raw message in Rsyslog and forward it to another remote machine or write it to a file? Or, how can I change the default header config to add a custom processname tag(if there is default config applied)?

meuh meuh · Accepted Answer · 2020-07-17T17:04:58

rsyslog is probably using the RSYSLOG_TraditionalForwardFormat default template for omfwd which is:

template(name="RSYSLOG_TraditionalForwardFormat" type="string"
     string="<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32% %msg:::sp-if-no-1st-sp%%msg%")

Define your own template and use it in your forwarding rule, eg:

template(name="myfwd" type="string"
     string="<%PRI%>%TIMESTAMP% %HOSTNAME% dummytag %rawmsg%\n")
if $fromhost == 'thehost' then {
    action(type="omfwd" target="theserver" protocol="tcp" template="myfwd")
}

Rsyslog parsing issue - no header in syslog message that is sent to Rsyslog

1 Answers