ElasticSearch only query nested if it exists

3
votes

I have this search query to find the query search term "red dog" in the root Title and Description and also match the nested comments document.

GET /_all/video/_search
{
   "query":{
      "bool":{
         "should":[
            {
               "multi_match":{
                  "query":"red dog",
                  "fields":[
                     "Title",
                     "Description"
                  ],
                  "type": "cross_fields",
                  "operator":"and"
               }
            },
        {
         "nested":{
                  "path":"Comments",
                  "query":{
                     "multi_match":{
                        "query":"red dog",
                        "fields":[
                            "Comments.Description",
                            "Comments.Description.folded"
                        ],
                        "type": "cross_fields",
                        "operator":"and"
                     }
                  }
               }
            }
         ]
      }
   }
}

Unfortunately for me, comments are sometimes null when I persist them to ElasticSearch, is it possible to do some sort of "include if document exists" condition?

Update

I still get the same error 

[nested] failed to find nested object under path [Comments]

When I try to query using exists

    GET /_all/video/_search
    {
       "query":{
          "bool":{
             "should":[
                {
                   "multi_match":{
                      "query":"lisa",
                      "fields":[
                         "Title",
                         "Description"
                      ],
                      "type":"cross_fields",
                      "operator":"and"
                   }
                },
                {
                   "nested":{
                      "path":"Comments",
                      "query":{
                         "filtered":{
                            "query":{
                               "match_all":{}
                            },
                            "filter":{
                               "exists":{
                                  "field":"Comments.Description"
                               }
                            }
                         }
                      }
                   }
                }
             ]
          }
       }
    }

My mapping for everything

{
   "en":{
      "mappings":{
         "video":{
            "properties":{
               "Comments":{
                  "type":"nested",
                  "properties":{
                     "Description":{
                        "type":"string",
                        "analyzer":"english",
                        "fields":{
                           "folded":{
                              "type":"string",
                              "analyzer":"folding"
                           }
                        }
                     }
                  }
               },
               "Description":{
                  "type":"string",
                  "analyzer":"english",
                  "fields":{
                     "folded":{
                        "type":"string",
                        "analyzer":"folding"
                     }
                  }
               },
               "Title":{
                  "type":"string",
                  "analyzer":"english",
                  "fields":{
                     "folded":{
                        "type":"string",
                        "analyzer":"folding"
                     }
                  }
               }
            }
         }
      }
   }
}

And my settings

{
    "settings": {
        "analysis": {
            "analyzer": {
                "folding": {
                    "tokenizer": "standard",
                    "filter": [
                        "lowercase",
                        "asciifolding"
                    ]
                }
            }
        }
    }
}
2
elasticsearch
exists filter should do what you want. elasticsearch.org/guide/en/elasticsearch/reference/current/… - jhilden
Mark, can you post your full mapping please. - jhilden
Done, please see bottom of original post. Also included my original settings - Mark Walsh
It may be an issue with your your mapping is defined. I created the following gist and it runs the query without any errors (on ES 1.4.4) gist.github.com/jayhilden/899112a5d0fe09c5f1e9 - jhilden
I've just checked an it does work on the "en" index. It doesn't work across all indexes ("_all"). Do you know why this is? Do you need to setup a default mapping across all indices? - Mark Walsh

2 Answers

0
votes

It turns out I had a marvel index which didn't have the mapping for my type in. I deleted the plugin and it was fine searching across all indexes.

0
votes

Easiest way to do this is to "denormalize" your data so that you have a property that contains the count and a boolean if it exists or not. Then you can just search on those properties.

For example:

{
   "id": 31939,
   "hasAttachments": true,
   "attachmentCount": 2,
   "attachments": [
      {
         "type": "Attachment",
         "name": "txt.txt",
         "mimeType": "text/plain"
      },
      {
         "type": "Inline",
         "name": "jpg.jpg",
         "mimeType": "image/jpeg"
      }
   ]  
}