I have most recent logstash installed (2.3), and I want to parse syslog using grok filter. So, here is the filter:
filter {
if [type] == "linux-syslog" {
grok {
match => { "message" => "^%{SYSLOGTIMESTAMP:syslog_timestamp}\s*%{SYSLOGHOST:syslog_hostname}\s*(%{PROG:syslog_program})?\s*(:?\[%{POSINT:syslog_pid}\])?:?\s*%{GREEDYDATA:syslog_message}[a-z]*\s*$" }
patterns_dir => ["/var/opt/logstash/patterns"]
add_tag => "syslog_everything"
keep_empty_captures => "true"
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
This is what I get from logstash trying to parse it:
"message" => "Apr 20 14:31:35 node1 ansible-service: Invoked with name=logstash pattern=None enabled=True state=restarted sleep=None arguments= runlevel=default ", "@version" => "1", "@timestamp" => "2016-04-20T14:31:35.000Z", "path" => "/var/log/syslog", "host" => "node1", "type" => "linux-syslog", "syslog_timestamp" => "Apr 20 14:31:35", "syslog_hostname" => "node1", "syslog_program" => "ansible-service:", "syslog_pid" => nil, "syslog_message" => "Invoked with name=logstash pattern=None enabled=True state=restarted sleep=None arguments= runlevel=default ", "tags" => [ [0] "syslog_everything", [1] "_grokparsefailure" ]}
And...
{ "message" => "Apr 20 14:35:10 node1 crontab[29052]: (vagrant) END EDIT (vagrant)", "@version" => "1", "@timestamp" => "2016-04-20T14:35:10.000Z", "path" => "/var/log/syslog", "host" => "node1", "type" => "linux-syslog", "syslog_timestamp" => "Apr 20 14:35:10", "syslog_hostname" => "node1", "syslog_program" => "crontab", "syslog_pid" => "29052", "syslog_message" => "(vagrant) END EDIT (vagrant)", "tags" => [ [0] "syslog_everything", [1] "_grokparsefailure" ] }
What am I doing wrong here? I checked this using grokdebugger and it passed fine...
