Parsing log lines for Kibana without logstash

Question

We have a bunch of log lines from various sources (in different formats) in elasticsearch v1.3. We don't have control over the logstash config that puts them in there, so it's basically just one field containing the log line.

Using simple queries looking for string matches, we've been able to do some pretty cool things in Kibana (v3), but I'd love to do some of the things we'd be able to easily do if the data were more structured (aggregates based on extracted numeric values, topN lists of substrings of the log lines, etc).

Is there a way to have Kibana or Elasticsearch do some simple parsing of the log lines?

You could do something with scripted fields (which you'd need to save on disk on the elasticsearch server), but the real answer is to gain control of the logstash config. Could you bring up your own logstash, and get the existing logstash to output to you? — Alain Collins

thikonom thikonom · Accepted Answer · 2015-05-06T16:14:04

Elasticsearch allows for Script Fields

Kibana 4 has a place for that in its interface which makes it easier. Navigate to Settings -> Indices -> Scripted Fields -> Create Scripted Field

Parsing log lines for Kibana without logstash

1 Answers