How to highlight nested object hits in Elasticsearch v5

Question

I am trying to get highlighting returned on hits from nested queries. According to the documentation, this is possible. The documentation says:

The parent/child and nested features allow the return of documents that have matches in a different scope. In the nested case, documents are returned based on matches in nested inner objects. The inner hits feature returns per search hit in the search response additional nested hits that caused a search hit to match in a different scope. Inner hits can be used by defining an inner_hits definition on a nested, has_child or has_parent query and filter. Inner hits also supports the following per document features:

'Highlighting', 'Explain Source', 'filtering', 'Script fields', 'Doc value', 'fields', 'Include versions'.

https://www.elastic.co/guide/en/elasticsearch/reference/5.0/search-request-inner-hits.html#nested-inner-hits

I am however unable to get this working based on the below mapping and query. Can anyone explain why and where I am going wrong?

NOTE: I am specifically referring to nested types and not parent_child.

I've found it only works when changing referenceIdValue and ReferenceIdType from keyword to text but obviously this is not what I want. ES Documentation says:

"The field name supports wildcard notation. For example, using comment_* will cause all text and keyword fields (and string from versions before 5.0) that match the expression to be highlighted. Note that all other fields will not be highlighted." https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-highlighting.html#_highlight_query I'm thinking this may be a bug?

Many thanks.

Mapping:

{
"entity": {
    "dynamic_templates": [{
        "catch_all": {
            "match_mapping_type": "*",
            "mapping": {
                "type": "text",
                "store": true,
                "analyzer": "phonetic_index",
                "search_analyzer": "phonetic_query"
            }
        }
    }],
    "_all": {
        "enabled": false
    },
    "properties": {
        "entityName": {
            "type": "text",
            "store": true,
            "analyzer": "indexed_index",
            "search_analyzer": "indexed_query",
        "referenceIds": {
            "type": "nested",
            "properties": {
                "referenceIdValue": {
                    "type": "keyword",
                    "norms": true,
                    "store": true
                },
                "referenceIdType": {
                    "type": "keyword",
                    "store": true,
                    "norms": true
                }
            }
        },
        "isPublished": {
            "type": "boolean",
            "store": true
        }
    }
}
}

Query:

curl -XGET "127.0.0.1:9200/test/entity/_search?pretty" -d"
{
"query": {
    "bool": {
        "filter": {
            "term": {
                "isPublished": "true"
            }
        },
        "must": [
            [{
                "nested": {
                    "query": {
                        "bool": {
                            "must": [
                                [{
                                    "term": {
                                        "referenceIds.referenceIdValue": "123456"
                                    }
                                }, {
                                    "term": {
                                        "referenceIds.referenceIdType": "ENCO"
                                    }
                                }]
                            ]
                        }
                    },
                    "inner_hits": {
                        "highlight": {
                            "fields": {
                                "referenceIds.referenceIdValue": {},
                                "referenceIds.referenceIdType": {}
                            }
                        }
                    },
                    "path": "referenceIds"
                }
            }]
        ]
    }
}
}"

result:

{
"took": 99,
"timed_out": false,
"_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
},
"hits": {
    "total": 1,
    "max_score": 14.319202,
    "hits": [{
        "_index": "test",
        "_type": "entity",
        "_id": "3423631",
        "_score": 14.319202,
        "_source": {
            "entityName": "Test Entity",
            "referenceIds": [{
                "referenceIdValue": "AAAABBBB",
                "referenceIdType": "SULA"
            }, {
                "referenceIdValue": "123456",
                "referenceIdType": "ENCO"
            }]
        },
        "inner_hits": {
            "referenceIds": {
                "hits": {
                    "total": 1,
                    "max_score": 14.319202,
                    "hits": [{
                        "_nested": {
                            "field": "referenceIds",
                            "offset": 3
                        },
                        "_score": 14.319202,
                        "_source": {
                            "referenceIdValue": "123456",
                            "referenceIdType": "ENCO"
                        }
                    }]
                }
            }
        }
    }]
}
}

haltabush haltabush · Accepted Answer · 2017-09-18T23:29:15

I think it's because in the highlighting part of your query you need to set the path relative to the nested document. Try this:

                "inner_hits": {
                    "highlight": {
                        "fields": {
                            "referenceIdValue": {},
                            "referenceIdType": {}
                        }
                    }
                },

How to highlight nested object hits in Elasticsearch v5

1 Answers