Elasticsearch nested significant terms aggregation with background filter

Question

I am having hard times applying a background filter to a nested significant terms aggregation , the bg_count is always 0.

I'm indexing article views that have ids and timestamps, and have multiple applications on a single index. I want the foreground and background set to relate to the same application, so I'm trying to apply a term filter on the app_id field both in the boo query and in the background filter. article_views is a nested object since I want to be also able to query on views with a range filter on timestamp, but I haven't got to that yet.

Mapping:

    {
    "article_views": {
        "type": "nested",
        "properties": {
            "id": {
                "type": "string",
                "index": "not_analyzed"
            },
            "timestamp": {
                "type": "date",
                "format": "strict_date_optional_time||epoch_millis"
            }
        }
    },
    "app_id": {
        "type": "string",
        "index": "not_analyzed"
    }
}

Query:

  {
   "aggregations": {
      "articles": {
         "nested": {
            "path": "article_views"
         },
         "aggs": {
            "articles": {
               "significant_terms": {
                  "field": "article_views.id",
                  "size": 5,
                  "background_filter": {
                     "term": {
                        "app_id": "17"
                     }
                  }
               }
            }
         }
      }
   },
   "query": {
      "bool": {
         "must": [
            {
               "term": {
                  "app_id": "17"
               }
            },
            {
               "nested": {
                  "path": "article_views",
                  "query": {
                     "terms": {
                        "article_views.id": [
                           "1",
                           "2"
                        ]
                     }
                  }
               }
            }
         ]
      }
   }
}

As I said, in my result, the bg_count is always 0, which had me worried. If the significant terms is on other fields which are not nested the background_filter works fine.

Elasticsearch version is 2.2.

Thanks

You seem to be hitting the following issue where in your background filter you'd need to "go back" to the parent context in order to define your background filter based on a field of the parent document. You'd need a reverse_nested query at that point, but that doesn't exist. One way to circumvent this is to add the app_id field to your nested documents so that you can simply use it in the background filter context. — Val
@Val Thanks, it seems like the right answer. If you want right it as an answer and I will accept it. — Orr

Val Val · Accepted Answer · 2016-12-21T13:09:03

You seem to be hitting the following issue where in your background filter you'd need to "go back" to the parent context in order to define your background filter based on a field of the parent document.

You'd need a reverse_nested query at that point, but that doesn't exist.

One way to circumvent this is to add the app_id field to your nested documents so that you can simply use it in the background filter context.

Elasticsearch nested significant terms aggregation with background filter

1 Answers