Loglevel pattern in GROK

1
votes

i'm new to logstash and grok and have a question regarding a pattern. I'm trying to create a grok pattern for my apache error logs. However i'm a bit confused about the following behavior.

If i use the underlying match in http://grokdebug.herokuapp.com/:

%{LOGLEVEL:severity%{IPV4:clientip}%{GREEDYDATA:errormsg}

I get a no match on the following log snippet:

[Sun Apr 10 09:28:01 2016] [error] [client 111.11.111.111] File does not exist: /var/www/html/favicon.ico

Now all the patterns work if i run %{LOGLEVEL:severity} or %{IPV4:clientip}%{GREEDYDATA:errormsg} seperately. But if i try it together i get a "no match" return.

I think this is happening because the %{LOGLEVEL:severity} truncates the rest of the log but i cannot figure why.

Anybody have any suggestion as to what i'm doing wrong ?

Thank you in advance.

2
logstashlogstash-grok

2 Answers

3
votes

As with all unanchored regular expressions, once you start matching, you need to match everything in the string. In your pattern, you have not accounted for the spaces or brackets that exist in your sample. (That's also assuming that the lack of a } after "severity" was a copy&paste error).

1
votes

The grok pattern you wrote indeed does not match the log message, why dont you try it with dissect {} which is more performant than grok filters?

Your message is:

[Sun Apr 10 09:28:01 2016] [error] [client 111.11.111.111] File does not exist: /var/www/html/favicon.ico

So you could write a dissect filter like:

dissect {
  mapping => {
    "message" => "[%{raw_date}] [%{loglevel}] [client %{client_ip}] %{log_message}"
  }
}