Is it possible to rewrite fields when forwarding in rsyslog?

Question

I have a large collection of servers that send syslog messages to a central rsyslog server, which in turn sends them logstash. The issue that Im having is that the sending servers are sending the machine name as "localhost" or the 10.0.0.0/8 ip address of their internal eth0 nic, which is not a huge problem as the rsyslog server is splitting the log files based on the source IP, and it logging them to their server remote-server-1.2.3.4.log file. The problem happens when the rsyslog server sends (with *.* @logstash) to logstash then has none of the source IP information.

is it possible to have rsyslog alter the source machine name/IP so that it includes the source IP address, so that logstash can aggregate on that correct source IP?

This is my logstash config

input {
  udp {
  port => 514
  type => syslog
}
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}


output {
elasticsearch {
   hosts => "esearch1:443"
   ssl => "true"
   manage_template => false
}}

Thanks

I couldnt find a way to do this within syslog or with iptables in a way that didnt cause traffic loops, so I used samplicate with ./samplicate -d 5 -S -p 514 -s 10.3.1.183 10.3.1.183/1554 10.3.1.183/1555 — Paul L

Luv33preet Luv33preet · Accepted Answer · 2017-08-12T23:37:01

It is not possible in rsyslog to rewrite fields.

But you can easily get the system ip from which the message is generated.

There is a property of every message rsyslog gets, fromhost-ip. You can use that in your logstash config.

Maybe something like,

match => { "message" => "%{SYSLOG-IP:syslog_fromhost-ip} %{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }

Find more here about it, http://www.rsyslog.com/doc/master/configuration/properties.html#message-properties

Is it possible to rewrite fields when forwarding in rsyslog?

1 Answers