logstash grok filter-grok parse failure

Question

I have multiline custom logs which I am processing as a single line by the filebeat multiline keyword. Now this includes \n at the end of each line. This however causes grok parse failure in my logstsash config file. Can someone help me on this. Here is how all of them look like:

Please help me with the grok filter for the following line:

11/18/2016 3:05:50 AM : \nError thrown is:\nEmpty Queue\n*************************************************************************\nRequest sent is:\nhpi_hho_de,2015423181057,e06106f64e5c40b4b72592196a7a45cd\n*************************************************************************\nResponse received is:\nQSS RMS Holds Hashtable is empty\n*************************************************************************

Kulasangar Kulasangar · Accepted Answer · 2017-03-02T10:06:10

As @Mohsen suggested you might have to use the gsub filter in order to replace all the new line characters in your log line.

filter {
  mutate {
    gsub => [
      # replace all forward slashes with underscore
      "fieldname", "\n", ""         
    ]
  }
}

Maybe you could also do the above within an if condition, to make sure that there's no any grokparse failure.

if "_grokparsefailure" in [tags] or "_dateparsefailure" in [tags] {
    drop { }
}else{
  mutate {
    gsub => [
      # replace all forward slashes with underscore
      "fieldname", "\n", ""         
    ]
  }
}

Hope this helps!

logstash grok filter-grok parse failure

2 Answers