GROK Pattern filtering

0
votes

Hi I am new to logstash and grok filtering, I have a sample log like this:

1/11/2017 12:00:17 AM : Error thrown is: No Error

Request sent is: webMethod:GetOSSUpdatedOrderHeader|appCode:OSS|regionCode:EMEA|orderKeyList:|lastModifedDateTime:1/10/2017 11:59:13 PM|

I want to filter out the line separator which is a line full of ** (the last line) Also when I want to be able to capture entire line including ":" in one field. For example in the above log, webMethod:GetOSSUpdatedOrderHeader has to be captured in one field in my grok pattern. Is there a way to achieve this?? TIA. Please refer the attached image for the sample log message

1
logstashgrok

1 Answers

1
votes

A few tips:

  • Photos of logs are not a good way to offer someone an example, copy and paste the log
  • The Grok Debugger is a great way of building your own grok patterns

This should work for the sample log line you pasted in:

%{NOTSPACE:webMethod}\|%{NOTSPACE:appCode}\|%{NOTSPACE:regionCode}\|%{NOTSPACE:orderKeyList}\|%{NOTSPACE:lastModifedDateTime}

However, what you requested, probably isn't quite what you want, as you just want the field content in the result, not the name of the field as well. This should give you more sensible results:

webMethod:%{NOTSPACE:webMethod}\|appCode:%{NOTSPACE:appCode}\|regionCode:%{NOTSPACE:regionCode}\|orderKeyList:(?:%{NOTSPACE:orderKeyList}|)\|lastModifedDateTime:%{NOTSPACE:lastModifedDateTime}

You would then want to process the lastModifedDateTime field with the date filter to get the date stamp in a format logstash can save to.