Logstash grok filter - field value duplicated

Question

My logstash filter is configured as follows:

filter {
  grok {
    patterns_dir => ["/usr/share/logstash/pipeline/patterns/"]
    match => {
      "[message]" => "%{TIMESTAMP_ISO8601:timestamp} %{THREAD:thread} %{LOGLEVEL:level} %{LOGGER:logger} %{CONTEXT:context} - %{GREEDYDATA:message}"
    }
  }
  mutate {
    rename => { "[fields][index]" => "application" }
    rename => { "[host][name]" => "instance" }
    remove_field => ["@version","agent.ephemeral_id","agent","ecs","fields","input","tags"]
  }
}

Grok debugger suggests everything is fine, and for error line:

2020-10-28 05:14:41,282 [Worker-5] DEBUG Amount  - calculate operation: [1], useCurrencyCodeOfPosition: [false]

I am getting the below output:

{
  "level": "DEBUG",
  "logger": "Amount",
  "context": "",
  "thread": "Worker-5",
  "message": "calculate operation: [1], useCurrencyCodeOfPosition: [false]",
  "timestamp": "2020-10-28 05:14:41,282"
}

Patterns are defined as follows:

THREAD \[(?<thread>[^\]]*)\]
LOGGER (?<logger>[^ ]*)
CONTEXT (?<context>[^-]*)

Now, each value produced by grok filter is duplicated as the below example shows:

             "logger" => [
    [0] "Amount",
    [1] "Amount"
],
             "thread" => [
    [0] "[Worker-5]",
    [1] "Worker-5"

What's the issue here? I just cannot figure it out. It's my first filter :). I'm working with Logstash 7.9.2 (dockerized)

This must have sth to do with my custom patterns... just do not know what yet :/ — m0rt1m3r
"In-lining" the patterns work as expected, but it's not the preferred way I'd think :/ — m0rt1m3r

karan shah karan shah · Accepted Answer · 2020-10-28T18:28:34

I think there are problems with the custom patterns in the filter. What you want can also be achieved simply using out of the box patterns like below

filter{
 grok {
    match => {
      "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}\[%{DATA:thread}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}%{NOTSPACE:logger} %{DATA:context}-%{SPACE}%{GREEDYDATA:message}"
    }
    overwrite => [ "message" ]
  }
  mutate {
    rename => { "[fields][index]" => "application" }
    rename => { "[host][name]" => "instance" }
    remove_field => ["@version","agent.ephemeral_id","agent","ecs","fields","input","tags"]
  }
}

Checkout this link of default grok patterns. If you need to do time-series analysis on these events, I would suggest you to override @timestamp with timestamp or at-least apply date filter on timestamp.

If you are expecting to capture multi-line stack trace errors, consider using multi-line filter on the input plugin.

Logstash grok filter - field value duplicated

1 Answers