Elasticsearch - aggregate over filtered data

Question

I have a query that returns a set of documents (100). Over these I want to apply an aggregation, because these are most relevant. When I try to aggregate, that returns aggregations over all results, not over the first 100.

Query:

{
  "size": 100,
  "sort": [
   {
    "_score": {
    "order": "desc"
    }
   }
],
"from": 0,
"query": {
 .......
 },
  "aggregations": {
    "category.category_id": {
      "nested": {
        "path": "category"
       },
    "aggregations": {
     "category.category_id": {
       "terms": {
         "field": "category.category_id",
         "size": 2,
         "order": {
           "_count": "desc"
         }
       }
     }
   }
 }
}

Result:

{
"took": 33,
"timed_out": false,
"_shards": {
 "total": 4,
 "successful": 4,
 "skipped": 0,
 "failed": 0
},
"hits": {
 "total": 1042,
 "max_score": 60,
 "hits": [...100 hits...]
},
"aggregations": {
 "category.category_id": {
  "doc_count": 5186,
  "category.category_id": {
   "doc_count_error_upper_bound": 0,
   "sum_other_doc_count": 196,
   "buckets": [
   {
     "key": 2,
     "doc_count": 1042
   },
   {
    "key": 2764,
    "doc_count": 272
   }
   ....
   ]
  }
 }
}

Expected:

{
"took": 33,
"timed_out": false,
"_shards": {
 "total": 4,
 "successful": 4,
 "skipped": 0,
 "failed": 0
},
"hits": {
 "total": 1042,
 "max_score": 60,
 "hits": [...100 hits...]
},
"aggregations": {
 "category.category_id": {
  "doc_count": 100,
  "category.category_id": {
   "doc_count_error_upper_bound": 0,
   "sum_other_doc_count": x,
   "buckets": [
   {
     "key": 2,
     "doc_count": x (x< 100) (eg 37)
   },
   {
    "key": 2764,
    "doc_count": y (y <= 100 -x) (eg 10)
   }
   ....
   ]
  }
 }
}

Is possible to aggregate over filtered data? or haw can I aggregate over most relevant data?

Kartik Puri Kartik Puri · Accepted Answer · 2020-02-05T18:51:16

You can use a filter aggregation as described by elasticsearch documentation

{
    "aggs" : {
        "agg_name" : {
            "filter" : { //Add your query },
            "aggs" : {
               "terms": {
                       "field": "category.category_id",
                      "size": 2,
                    "order": {
                     "_count": "desc"
                      }
            }
        }
    }
}

If you want you can add one more aggregation inside the 2nd aggs

Elasticsearch - aggregate over filtered data

1 Answers