Spring Data Elasticsearch doesn't implicitly create mapping from annotations

Question

I'm trying to work with Spring Data Elasticsearch 4.0.1 and trying to figure out how and when the Elasticsearch mappings are being created by Spring or if they are being created by Spring.

If I have an entity like so:

@Document(indexName = "companies")
public class CompanyEntity {

  @Id
  private final String id;

  @MultiField(
      mainField = @Field(type = Text),
      otherFields = {
          @InnerField(suffix = "raw", type = Keyword)
      }
  )
  private final String companyName;

  @PersistenceConstructor
  public CompanyEntity(String id, String companyName) {
    this.id = id;
    this.companyName = companyName;
  }

  public String getId() {
    return id;
  }

  public String getCompanyName() {
    return companyName;
  }
}

I was under the impression that Spring would implicitly create the mapping for this index but I seem to be mistaken. Elasticsearch still creates the mapping for this index.

{
    "companies": {
        "mappings": {
            "properties": {
                "_class": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "companyName": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "id": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}

Clearly from the above, the suffix in the InnerField annotation is not used and the ignore_above value is not used as well since it defaults to -1, and Elasticsearch would remove this field altogether if trying to set ignore_above to -1.

The only way I was able to get the mapping for the above annotations was to explicitly set the mapping myself.

@Autowired private ElasticsearchOperations operations;

Document mapping = operations.indexOps(CompanyEntity.class).createMapping();
operations.indexOps(CompanyEntity.class).putMapping(mapping);

Which yields the expected mapping:

{
    "companies": {
        "mappings": {
            "properties": {
                "_class": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "companyName": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "keyword"
                        }
                    }
                },
                "id": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}

Which is fine, but feels a bit odd to me as I could not find any details of this approach in the official Spring Data Elasticsearch docs. And the JavaDocs are kinda void of any details.

Is this the right approach to installing mappings into Elasticsearch from Spring Data?

1. Created a sample project with sprint-data-elastic (3.2.9.RELEASE) and defined the document mapping with a multi-field (text, keyword) and ran the project. The mapping generated is what I expect. Can you please share the version of spring-data-elastic & elastic-search which you are using. — Sahil Gupta
2. You can set a debug point on ElasticsearchRestTemplate -> "putMapping(String indexName, String type, Object mapping)" to see what mapping is generated. — Sahil Gupta
3. Is this the right approach to installing mappings into Elasticsearch from Spring Data? - Es is not like Mysql where-in you would get an exception if the field doesn't exist in the store, rather it creates a new field if not found. Hence it is alright to install mappings via spring data as long as you define it correctly and the generated mapping is what you intend to. Am i able to answer all your questions ? — Sahil Gupta
@SahilGupta, I the version I am using is 4.0.1. And the expected mapping is the last mapping I detail in my post. The first mapping I detail is the mapping that Elasticsearch creates automatically. And I will try to do a bit of debugging on Spring's side to see if any mappings are being created. Regarding your last point, I understand that I might have to create the mapping manually, but this does break expectations of using Spring-Data. I'm trying to understand if I need to always create these mappings manually as this detail is not in the official spring docs. — FlatPenguin

P.J.Meisch P.J.Meisch · Accepted Answer · 2020-08-21T05:32:05

If you are using the ElasticsearchOperations to work with, then this is the right way to create the mapping. In version 4.1 there will be an additional method

boolean putMapping(Class<?> clazz)

which combines the two steps of creating and writing the mapping.

ElasticsearchOperations and IndexOperations - or better the implementations - are the more low level access to Elasticsearch in Spring Data Elasticsearch - create an index, put the mapping, write an entity etc. These are elemantary operations. You have the full control which operations are executed, but it's your respsonsibility to execute them.

Building on top of that is the repository support. If you define a repository interface

interface CompanyRepository extends ElasticsearchRepository<CompanyEntity, String>{}

which you inject in one of your classes

@Autowired CompanyRepository repository;

then on application startup Spring Data Elasticsearch will create an implementation of this interface and will check if the index defined by the entity's @Document annotation exists. If not, it will create the index and write the mapping - The @Document annotation has an argument createIndex which by default is true.

So for autocreation you must use the repository, the repository support also makes use of the ElasticsearchOperations to offer things like query derivation from method names, paging support and more.

Spring Data Elasticsearch doesn't implicitly create mapping from annotations

1 Answers