Aggregation, Query Context and filter Context not working in Elasticsearch 5.1

0
votes

I am facing issue in migrating from elastic search 1.5 to 5.1. Following is my elastic search - 1.5 Query:

{
    "_source":["_id","spotlight"],
    "query":{
        "filtered":{
            "filter":{
                "and":[
                    {"term":{"gender":"female"}},
                    {"range":{"lastlogindate":{"gte":"2016-10-19 12:39:57"}}}
                ]
            }
        }
    },
    "filter":{
        "and":[
            {"term":{"maritalstatus":"1"}}
        ]
    },
    "sort":[{"member2_dummy7":{"order":"desc"}}],
    "size":"0",
    "aggs": {

        "maritalstatus": {

            "filter": {},
            "aggs" : {

                "filtered_maritalstatus": {"terms":{"field":"maritalstatus","size":5000}}
            }
        }
    }
}

This query is giving me correct doc_count in aggregations. This doc_count is calculated over result set returned by query context and it ignores filter context.

I have written same query in elastic search 5.1:

{
    "_source":["_id","spotlight"],
    "query":{
        "bool":{
            "must":[
                {"term":{"gender":"female"}},
                {"range":{"lastlogindate":{"gte":"2016-10-19 12:39:57"}}}
            ],
            "filter":{
                "bool":{
                    "must":[
                        {"term":{"maritalstatus":"1"}}
                    ]
                }
            }
        }
    },
    "sort":[{"member2_dummy7":{"order":"DESC"}}],
    "size":"0",
    "aggs": {

        "maritalstatus": {

            "filter": {},
            "aggs" : {

                "filtered_maritalstatus": {"terms":{"field":"maritalstatus","size":5000}}
            }
        }
    }

}

But in elastic search 5.1, it is returning me wrong doc_count in aggregation. I think it is taking filter in query context and hence, it is returning wrong doc_cout. Can someone tell me correct way to separate query and filter in elastic search 5.1?

1
elasticsearchaggregationelasticsearch-5
Check you first term query in 1.5 you were looking for female and in 5.1 you were searching for Female. It matters a lot in term query. So change your term query in 5.1 from Female to female and check the results. Also have a look at timestamp the on in 5.1 query is 5 seconds greater than the one in 1.5 query. – avr

1 Answers

2
votes

Your 1.5 query uses post_filter which you have removed in your 5.1 query.

The equivalent query in ES 5.1 is the following (filtered/filter simply gets replaced as bool/filter and the top-level filter renamed to post_filter):

{
  "_source": [
    "_id",
    "spotlight"
  ],
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "gender": "female"
          }
        },
        {
          "range": {
            "lastlogindate": {
              "gte": "2016-10-19 12:39:57"
            }
          }
        }
      ]
    }
  },
  "post_filter": {
    "term": {
      "maritalstatus": "1"
    }
  },
  "sort": [
    {
      "member2_dummy7": {
        "order": "desc"
      }
    }
  ],
  "size": "0",
  "aggs": {
    "maritalstatus": {
      "filter": {},
      "aggs": {
        "filtered_maritalstatus": {
          "terms": {
            "field": "maritalstatus",
            "size": 5000
          }
        }
      }
    }
  }
}