grok filter not matching syslog message

Question

After searching around a bit and trying out various solutions, this is my grok pattern:

%{TIME}", "interface", "%{EMAILLOCALPART}","MAC", "%{IP}:%{MAC}", "src", "SRC=%{IP}", "dst", "DST=%{IP}

But it is not matching my syslog message. The messages originate from IPFire and look like this:

00:00:06 IN=red0 OUT= MAC=00:0d:b9:42:65:fc:00:17:10:82:5f:00:08:00 SRC=179.43.177.194 DST=46.127.208.49 LEN=60 TOS=0x00 PREC=0x00 TTL=56 ID=62301 DF PROTO=TCP SPT=37305 DPT=6666 WINDOW=29200 RES=0x00 SYN URGP=0 
00:00:07 IN=red0 OUT= MAC=00:0d:b9:42:65:fc:00:17:10:82:5f:00:08:00 SRC=116.31.116.17 DST=46.127.208.49 LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=4401 DF PROTO=TCP SPT=23103 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0 
00:00:07 IN=red0 OUT= MAC=00:0d:b9:42:65:fc:00:17:10:82:5f:00:08:00 SRC=74.79.43.231 DST=46.127.208.49 LEN=131 TOS=0x00 PREC=0x00 TTL=113 ID=16393 PROTO=UDP SPT=36303 DPT=6666 LEN=111 
00:00:08 IN=red0 OUT= MAC=00:0d:b9:42:65:fc:00:17:10:82:5f:00:08:00 SRC=84.241.202.2 DST=46.127.208.49 LEN=129 TOS=0x00 PREC=0x00 TTL=51 ID=41009 DF PROTO=UDP SPT=35858 DPT=1025 LEN=109 
00:00:09 IN=red0 OUT= MAC=00:0d:b9:42:65:fc:00:17:10:82:5f:00:08:00 SRC=198.8.80.183 DST=46.127.208.49 LEN=132 TOS=0x00 PREC=0x00 TTL=117 ID=21082 PROTO=UDP SPT=24883 DPT=1025 LEN=112

I am also not seeing any of my defined fields in Kibana. I have a very basic config file (compared to others), but I can't seem to wrap my head around it. I would need to see a proper example to understand it properly if anyone could help.

My 11-ipfire config:

input {
    udp {
    port => 5514
    }
    tcp {
    port => 5514
    }
}
filter {
    if [type] == "syslog" {
        grok {
            match => [ "time", "%{TIME}", "interface", "%{EMAILLOCALPART}","MAC", "%{IP}:%{MAC}", "src", "SRC=%{IP}", "dst", "DST=%{IP}" ]
        }
    }
}
output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "ipfire"
    }
}

Will Barnwell Will Barnwell · Accepted Answer · 2016-08-12T19:01:05

The grok filter's match takes a hash { "field" => "pattern" } or if one field might have different contents you can pass an array and grok will sequentially try matching the field to each array element { "field" => ["pattern1","pattern2", "patternX"] }. You are currently passing match an array, which will definitely not parse your example logs, or anything.

The grok pattern you supplied would match a literal string

"12:34", "interface", "bob.saget67", "MAC", "127.0.0.1:12-34-56-78-9A-BC", "src", "SRC=127.0.0.2", "dst", "DST=127.0.0.3"

I'm not going to write your grok pattern for you, but you can easily develop and test a grok pattern using online tools such as grok constructor. Additionally, I beleive Logstash comes with some pre-baked syslog patterns.

grok filter not matching syslog message

1 Answers