grok pattern to parse the logs using logstash

Question

I have been trying to parse a sample log file using logstash grok filter but was unable to output the distinguish fields. my sample logs look like following-

INFO  [2016-05-26 11:54:57,741]  [main]: org.eclipse.jetty.util.log:?:?- Logging initialized @5776ms`enter code here`

what i want to separate out is INFO, timestamp ,[main] and the message in two parts from from ?:?. what pattern i have tried in grok filter is ->

match => { "message" => "%{WORD:severity} %{CISCOTIMESTAMP:timestamp} %{NOTSPACE} %{GREEDYDATA:logmsg}" }

but its not correctly output the pattern. can please someone provide me the correct grok pattern match!! Any related help would be useful!!

doz10us doz10us · Accepted Answer · 2016-07-26T18:08:58

As it is not clear what exact format do you want to get, I provide you with following filter:

match => { "message" => "%{LOGLEVEL:severity} *\[%{TIMESTAMP_ISO8601:timestamp}\] *\[%{WORD:tread}\]\: *%{NOTSPACE:file} *%{GREEDYDATA:msg}" }

This will effectively split your example to:

{
  "severity": [
    [
      "INFO"
    ]
  ],
  "timestamp": [
    [
      "2016-05-26 11:54:57,741"
    ]
  ],
  "YEAR": [
    [
      "2016"
    ]
  ],
  "MONTHNUM": [
    [
      "05"
    ]
  ],
  "MONTHDAY": [
    [
      "26"
    ]
  ],
  "HOUR": [
    [
      "11",
      null
    ]
  ],
  "MINUTE": [
    [
      "54",
      null
    ]
  ],
  "SECOND": [
    [
      "57,741"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "tread": [
    [
      "main"
    ]
  ],
  "file": [
    [
      "org.eclipse.jetty.util.log:?:?-"
    ]
  ],
  "msg": [
    [
      "Logging initialized @5776ms`enter code here`"
    ]
  ]
}

This doesn't gracefully parse :?:?- part, so adjust it if needed.

Take a look at Grokdebug which is great for on-the-fly filter testing.

grok pattern to parse the logs using logstash

1 Answers