Hibernate Search with Elasticsearch creating fields with .keyword suffix

Question

I just implemented the integration of Hibernate Search with Elasticsearch using hibernate search 5.8 and ES 5.5. I have several fields created specifically for sorting, and they are all called [field]Sort.

When I was testing it locally, the first time I let Hibernate create the indexes, it created the String sort fields like this:

nameSort -> text
nameSort.keyword -> keyword

I realized that I should use the suffixed field for sorting. But then, when I destroyed my Elasticsearch cluster, to start over, it didn't create the suffixed fields, it just created the sort fields as keyword directly. I recreated the cluster 5 or more times again and it never created the suffixed fields again.

When I finally sent my changes to our staging environment, it created the suffixed fields again, causing my queries to fail, because they are trying to sort by a text field, instead of a keyword field.

Now, I'm really not sure of why it sometimes creates the suffix and sometimes doesn't. Is there any rule? Is there a way to avoid it creating 2 fields and making it always create only one keyword field with exactly the name I gave it?

Here's an example of a sort field:

@Field(name = "nameSort", analyze = Analyze.NO, store = Store.YES, index = Index.NO)
@SortableField(forField = "nameSort")
public String getNameSort() {
    return name != null ? name.toLowerCase(Locale.ENGLISH) : null;
}

Thanks in advance for any help.

yrodiere yrodiere · Accepted Answer · 2017-09-07T15:06:29

Hibernate Search does no such thing as creating a separate keyword field for text fields. It creates either a text field or a keyword field, depending on whether the field should be analyzed. In your case, the field is not analyzed, so it should create a keyword field.

Now, Hibernate Search is not alone here, and this behavior could stem from the Elasticsearch cluster itself. Did you check whether you have particular index templates on your Elasticsearch cluster? It could lead to Elasticsearch creating a keyword field whenever Hibernate Search creates a text property.

On a side note, you may be interested by the fact Hibernate Search 5.8 allows defining normalizers (same thing as Elasticsearch normalizers), which would allow you to annotate the getName() getter directly and avoid doing the lowercase conversion yourself. See this blog post for more information.

Hibernate Search with Elasticsearch creating fields with .keyword suffix

1 Answers