Spring data Elasticsearch change indexName dynamically

Question

I'm trying to use spring data elastisearch save some data. I need to create same index for different client. Ex. If I have index my-index, I need create my-index-A, my-index-B for client A and B. But annotation @Document works only with static indexName or with spEL which is not thread safe.

My question is, if I create index and search manually (ElasticsearchTemplate.createIndex(), NativeSearchQueryBuilder().withIndices()), and delete this line on entity class.

@Document(indexName = "my-index-A")

The entity can still receive its values? In another words, the annotation

@Id
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String aid;

@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userId;

@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String entityId;

@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userName;

Still works?

ibexit ibexit · Accepted Answer · 2018-11-20T23:19:18

TL;DR

Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.

Explanation:

If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.

Solution

I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.

Proof

Writing with

    ElasticEntity foo = new ElasticEntity();
    foo.setAid("foo-a-id");
    foo.setEntityId("foo-entity-id");
    foo.setUserName("foo-user-name");
    foo.setUserId("foo-user-id");

    IndexQuery fooIdxQuery = new IndexQueryBuilder()
            .withIndexName("idx-foo")
            .withObject(foo)
            .build();

    String fooId = template.index(fooIdxQuery);

and

    ElasticEntity bar = new ElasticEntity();
    bar.setAid("bar-a-id");
    bar.setEntityId("bar-entity-id");
    bar.setUserName("bar-user-name");
    bar.setUserId("bar-user-id");

    IndexQuery barIdxQuery = new IndexQueryBuilder()
            .withIndexName("idx-bar")
            .withObject(bar)
            .build();

    String barId = template.index(barIdxQuery);

should store the objects in differnet indices.

Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 10,
    "successful" : 10,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "idx-bar",
        "_type" : "elasticentity",
        "_id" : "bar-a-id",
        "_score" : 1.0,
        "_source" : {
          "aid" : "bar-a-id",
          "userId" : "bar-user-id",
          "entityId" : "bar-entity-id",
          "userName" : "bar-user-name"
        }
      },
      {
        "_index" : "idx-foo",
        "_type" : "elasticentity",
        "_id" : "foo-a-id",
        "_score" : 1.0,
        "_source" : {
          "aid" : "foo-a-id",
          "userId" : "foo-user-id",
          "entityId" : "foo-entity-id",
          "userName" : "foo-user-name"
        }
      }
    ]
  }
}

As you can see, the index name and the _id is correct in the response.

Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)

SearchQuery searchQuery = new NativeSearchQueryBuilder()
              .withQuery(matchAllQuery())
              .withIndices("idx-foo", "idx-bar")
              .build();

List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());

The mapping works too as the logger yields fully populated classes in the result:

[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]

Hope this helped!

Spring data Elasticsearch change indexName dynamically

1 Answers