ElasticSearch: populating ip_range type field via logstash

Question

I'm experimenting with the ip_range field type in ElasticSearch 6.8 (https://www.elastic.co/guide/en/elasticsearch/reference/6.8/range.html) and struggle to find a way to load ip data into the field properly via logstash

I was able to load some sample data via Kibana Dev Tools, but cannot figure out a way to do the same via logstash.

Index definition

PUT test_ip_range
{
  "mapping": {
    "_doc": {
      "properties": {
        "ip_from_to_range": {
          "type": "ip_range"
        },
        "ip_from": {
          "type": "ip"
        },
        "ip_to": {
          "type": "ip"
        }
      }
    }
  }
}

Add sample doc:

PUT test_ip_range/_doc/3
{
  "ip_from_to_range" : 
  {
    "gte" : "<dotted_ip_from>",
    "lte": "<dotted_ip_to>"
  }
}

Logstash config (reading from DB)

input {
  jdbc {
  ...
  statement => "SELECT ip_from, ip_to, <???> AS ip_from_to_range FROM sample_ip_data"
  }
}
output {
  stdout { codec => json_lines }
  elasticsearch {
  "hosts" => "<host>"
  "index" => "test_ip_range"
  "document_type" => "_doc"
  }
}

Question:

How do I get ip_from and ip_to DB fields into their respective gte and lte parts of the ip_from_to_range via logstash config??

I know I can also insert the ip range in CIDR notation, but would like to be able to have both options - loading in CIDR notation and loading as a range.

akb akb · Accepted Answer · 2021-07-07T08:15:57

After some trial and error, finally figured out the logstash config.

I had posted about a similar issue here, which finally got me on the right track with the syntax for this use case as well.

input { ... }
filter {
  mutate {
    add_field => {
      "[ip_from_to_range]" => 
      '{
        "gte": "%{ip_from}",
        "lte": "%{ip_to}"
       }'
    }
  }
  json {
    source => "ip_from_to_range"
    target => "ip_from_to_range"
  }
}
output { ... }

Filter parts explained

mutate add_field: create a new field [ip_from_to_range] with its value being a json string ( '{...}' ). It is important to have the field as [field_name], otherwise the next step to parse the string into json object doesn't work
json: parse the string representation into a json object