0
votes

I'd like to visualize how our average elasticsearch document size is changing over time.

  1. We use ElasticSearch v7.1. We use the AWS ElasticSearch service and the Mapper Size Plugin is installed.
  2. I enabled the _size field on my index. ref
  3. In Kibana "Discover" I can find small documents using: _size: <900
    • Or I can query from the command line: curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{"query": {"range": {"_size": { "lt": 900 }}}}' | jq .

Now I want to create a Kibana visualization, for example a date histogram with the median _size, but Kibana "Visualize" won't let me select _size as the aggregation field. Is there a way to visualize the size of documents?


_size is a "meta-field".

_size The size of the _source field in bytes, provided by the mapper-size plugin.

Maybe "meta fields" aren't supported in Kibana?

I can aggregate by _size when not using Kibana "Visualize":

curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{
  "query": {
    "range": {
      "_size": {
        "gt": 10
      }
    }
  },
  "aggs": {
    "sizes": {
      "terms": {
        "field": "_size",
        "size": 10
      }
    }
  },
  "sort": [
    {
      "_size": {
        "order": "desc"
      }
    }
  ]
}' | jq .
1

1 Answers

1
votes

If you go in Kibana to Management->Elasticsearch index management you can see for specific index that:

enter image description here

Some fields does not have Aggretable on. Check other fields and see that they have circle in this column. And that's the reason why Kibana Visualize won't let you select _size as the aggregation field.

Maybe you could use https://www.elastic.co/guide/en/kibana/current/scripted-fields.html somehow to make it work by making additional field, if Kibana would allow you to use _size field inside of a script and that field would be Aggretable in a sense i mentioned earlier.