Logstash grok pattern for custom log text file

Question

I am new to logstash, and I am trying to create grok pattern for my log file which is of type text. The data logged in file is as follows:

Timestamp: 24-03-2016 19:59:11
Message: Received request to get data
Title:GetData()
Machine: LTPN
----------------------------------------
Timestamp: 24-03-2016 20:15:34
Message: ERROR [08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
ERROR [01000] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()).
ERROR [01S00] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute
Title:GetData() 
Machine: LTPN
----------------------------------------

I want to grok it in such a way that it should populate the below fields as:

Timestamp = 24-03-2016 20:15:34
Messsage = ERROR [08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
ERROR [01000] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()).
ERROR [01S00] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute
Title = GetData()
Machine = LTPN

Could someone please help me in creating the pattern?

Alain Collins Alain Collins · Accepted Answer · 2016-04-11T15:38:17

The first step is to combine your multi-line message into a single logstash event. Depending on how the data is getting to logstash, you might be able to do it on that end (filebeat supports multiline). If not, check out the multiline codec.

Once you have the lines combined, then you're off to the grok{} filter to apply a regular expression that will create the fields you want.

Logstash grok pattern for custom log text file

1 Answers