Elasticsearch nested object filter

Question

I am new to Elasticsearch and I am trying to create a filter to retrieve documents that have specific attributes.

The attributes are defined as nested objects in the mapping like so:

"attributes": {
  "type": "nested",
    "properties" : {
      "id": {
        "type": "integer"
      },
      ...
    }
  }
}

I am trying to perform a complex query in the form of:

(attribute.id == 1 OR attribute.id == 2) AND (attribute.id == 3 OR attribute.id == 4)

According to what I've read so far I've created the following query to es:

{
  "query": {
    "filtered": {
     "query": {
        "match_all": {}
     },
     "filter": {
        "nested": {
           "path": "attributes",
           "filter": {
              "bool": {
                "must": [
                    { "bool" : {
                      "should" : [
                        { "term": { "attributes.id": 1 }},
                        { "term": { "attributes.id": 2 }}
                      ]
                    }},
                    { "bool" : {
                      "should" : [
                        { "term": { "attributes.id": 3 }},
                        { "term": { "attributes.id": 4 }}
                      ]
                    }}
                ]
              }
           }
        }
     }
  }
},
"sort": {
    "date": { "order": "desc" }
  }
}

However this doesn't return any results. If I remove one of the two bools in the must block it filters the documents correctly.

Same problem exists (no results) if I change the query (for testing purposes) to:

"must": [
  { "term": { "attributes.id": 3 }},
  { "bool" : {
    "should" : [
      { "term": { "attributes.id": 1 }},
      { "term": { "attributes.id": 2 }}
    ]
  }}
]

which to my understanding translates to attributes.id == 3 AND (attributes.id == 1 OR attributes.id == 2)

This is elasticsearch 2.x. What I am doing wrong?

Perhaps each bool clause could be in its own nested clause? Instead of having it all wrapped up in one. — Evaldas Buinauskas
@EvaldasBuinauskas I will give it a try and get back to you. — mobius
Can you post a sample document you expect to be returned by this query? — goalie7960

BrookeB BrookeB · Accepted Answer · 2016-01-22T18:16:21

In order to get the results you are looking for, you will need to do two separate nested queries and link them together with a bool query.

Here is an example:

{
  "bool":{
    "must":[
      {
        "nested":{
          "path":"attributes",
          "filter":{
            "bool":{
              "should":[
                { "term": {"attributes.id": 1 }},
                { "term": {"attributes.id": 2 }}
              ]
            }
          }
        }
      },
      {
        "nested":{
          "path":"attributes",
          "filter":{
            "bool":{
              "should":[
                { "term": {"attributes.id": 3 }},
                { "term": {"attributes.id": 4 }}
              ]
            }
          }
        }
      }
    ]
  }
}

The reason for this is due to a subtle nuance with nested documents and nested queries. Have a look at the part of the documentation on nested queries that says:

The query is executed against the nested objects / docs as if they were indexed as separate docs (they are, internally) and resulting in the root parent doc (or parent nested mapping).

When doing a nested query, you are not actually performing your query against the root document (though it feels that way). The query is operating against the nested documents as if they were separate docs.

Therefore, when you query for a nested document that has both id=(1 or 2) AND id=(3 or 4) you get no results. Each nested document, in your data it seems, has only one of these id values.

Elasticsearch nested object filter

1 Answers