I am attempting to create GeoIP data using an ELK stack, which can be visualized in Kibana.
I have recently installed an ELK stack (Elastic Search, Logstash, and Kibana) on a virtual instance of Ubuntu Server 14.04. I am using Bro to capture logs.
Everything to do with capturing the logs, parsing them, and viewing them in Kibana is working great, except for GeoIP (one of the most interesting features!).
The GeoIP portion of my logstash config file looks like this;
geoip {
add_tag => [ "geoip" ]
database => "/etc/logstash/GeoLiteCity.dat"
source => "id.orig_h"
target => "geoip"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}"]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
I had gotten that part of the filter from the following guide to setup GeoIP with Kibana. I have also seen that filter used in a few other places. I have attempted to simplify the filter (just source, target, database), but no luck.
When I do a curl request for my index, especially with the add_tag ["geoip"] it returns blank data;
"geoip" : {
"dynamic" : "true",
"properties" : {
"location" : {
"type" : "geo_point"
}
}
Finally, here is some scrubbed data that I have taken directly from Kibana, in JSON format;
{
"_index": "logstash-2015.11.12",
"_type": "bro-conn_log",
"_id": "*****",
"_score": null,
"_source": {
"message": [
"*****"
],
"@version": "1",
"@timestamp": "2015-11-12T13:43:16.205Z",
"host": "elk",
"path": "/nsm/bro/logs/current/conn.log",
"type": "bro-conn_log",
"ts": "*****",
"uid": "*****",
"id.orig_h": "*****",
"id.orig_p": *****,
"id.resp_h": "*****",
"id.resp_p": *****,
"proto": "*****",
"service": "*****",
"duration": *****,
"orig_bytes": *****,
"resp_bytes": *****,
"conn_state": "*****",
"local_orig": "*****",
"missed_bytes": *****,
"history": "*****",
"orig_pkts": *****,
"orig_ip_bytes": *****,
"resp_pkts": *****,
"resp_ip_bytes": *****,
"tunnel_parents": "*****",
"column21": "(empty)",
"conn_state_full": "*****"
},
"fields": {
"@timestamp": [
1447335796205
]
},
"sort": [
1447335796205
]
}
To summarize: I am attempting to get GeoIP data working with an ELK stack. Despite following guides describing how to do exactly that, the GeoIP field will not display in Kibana. Any advice would be GREATLY appreciated.