Too many buckets, aggregation fails on histogram

2
votes

I have a query to elastic that contains total 8 nested aggregations, all aggregations are term aggregations except one which is a histogram aggregation. If I remove that histogram aggregation, the query runs perfectly. But with histogram aggregation it is throwing this particular error:

This aggregation creates too many buckets (10001) and will throw an error in future versions. You should update the [search.max_buckets] cluster setting or use the [composite] aggregation to paginate all buckets in multiple requests.

Now I tried increasing the max_buckets size but the size is reaching above 100000 and also that number is not certain so that option is out. Then I tried creating a composite aggregation as suggested in the error but that is also giving the same error with the histogram added.

So my question is am I writing the queries in some wrong fashion or elastic is not feasible enough to create that much of buckets?

Kibana query: (with only 3 levels of nesting, original problem have 8)

GET /project/test/_search
{
    "query": {
        "bool": {
            "must": [
                {
                "range": {
                    "date" : {
                        "gte": 20180101,
                        "lte": 20180630
                        }
                    }
                }
            ]
        }
    },
    "size": 0,
    "aggs": {
        "agg1": {
            "terms": {
                "field": "agg1"
            },
            "aggs": {
                "agg2": {
                    "terms": {
                        "field": "agg2"
                    },
                    "aggs": {
                        "agg3": {
                            "histogram": {
                                "field": "agg3",
                                "interval": 1
                            }
                        }
                    }
                }
            }   
        }
    }
}
1
elasticsearchkibana

1 Answers

4
votes

You can do this to change your cluster setting (search.max_buckets) larger. https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html

PUT _cluster/settings
{
  "persistent": {
    "search.max_buckets": 50000
  }
}