Kibana: Duration between logs and Calculated fields

Question

I am trying to use Kibana to calculate the duration between two log events. The issue I am having is that the logs do not have a unique identifier. I would have to group them based on certain columns. In the discovery tab I have it set up like this. I searched for "communication" and "drum sw" and sorted the results by time.

1:  September 10th 2015, 03:04:41.000   Communication   Failure drum    sw  939
2:  September 10th 2015, 03:04:46.000   Communication   Normal  drum    sw  939
3:  September 10th 2015, 03:28:07.000   Communication   Failure drum    sw  2305
4:  September 10th 2015, 03:28:13.000   Communication   Normal  drum    sw  2305

Is there a way to group these logs by their district(sw) and device type(SW)? In addition to that, how can I create a calculated field to determine the duration between the event?

So for the first two the duration between them is 5 seconds: Ex: (present time) - (end time) = 5 seconds. Is this something I would do in a scripted field?

I am using the latest version of the ELK configuration. Kibana 4.2 elasticsearch and logstash 2.0.0.

Thanks!

Answer 1

With the elapsed{} filter, you can specify a complex pattern that is your unique key (like combining two fields).

Answer 2

I think that with scripted field is impossible because:

There are a few limitations when using Lucene Expressions:

Stored fields are not available

With this i supose that you can not reference the time because is a store field that dont have an unique value. One solution perhaps is to store the time before of each date. One example:

1: September 10th 2015, 03:04:41.000 Communication Failure drum sw 939 and you added a camp name TimeBefore with value September 10th 2015, 03:04:46.000 , the with scripted field you do:

doc['Time'].value-doc['TimeBefore'].value and save this camp as difference times.

Perhaps there are other better solutions, but this is the only that i can imagine.

Luck with Kibana.