Elastic Search filtered aliases with unmapped fields

Question

I recently started playing around with filtered aliases in Elastic Search (documentation here) but there is use case which I am not sure how to approach.

Use Case

Each document that I index in ElasticSearch for a fact is going to have a field called "tenantId" (and also some other fields such as "type", "id" etc). Now all the documents reside in the same index, so per Tenant I want to make sure I create a filtered alias. Now I want to create the filtered alias as soon as I have created the tenant itself and have the "tenantId" handy.

Problem

When I tried to create the alias programmatically using their java client, I get the following exception:

Caused by: org.elasticsearch.index.query.QueryParsingException: 
     [mdm-master] Strict field resolution and no field mapping 
     can be found for the field with name [tenantId]

Researching more, I found out that I can probably use dynamic templates in order to achieve this. So I created a template, saved it under config/templates, recreated my index and tried the same thing again. Got the same exception again. On reading the documentation more here (bottom 3 lines on the page), I found out that even if I would try to change the following property index.query.parse.allow_unmapped_fields to true (which I didn't try yet), for filtered aliases, it will force it to false.

Now the question is, how do I approach my usecase? I do not know the mappings of the corresponding types but what I do know for a fact is each document that I index, regardless of the type, will ALWAYS have a field called tenantId and that's what I want to create my filtered alias on.

EDIT

Couple of helpful links that I found. Not sure which version is this fixed on. filtered aliases in templates do not inherit mappings from aliased index #8473 index.query.parse.allow_unmapped_fields setting does not seem to allow unmapped fields in alias filters #8431

SECOND EDIT

Found an open bug for ElasticSearch with the exact same problem. Waiting on ES developers for a response. Failure to create Filtered Alias on empty index with template mappings #10038

All help is extremely appreciated ! I have been trying to figure this out from couple of days now with no luck :(.

Following is the code I used to add the filtered alias, and the default mapping json template

Template

{
  "template-1": {
    "template": "*",
    "mappings": {
      "_default_": {
        "properties": {
          "type": {
            "type": "string",
            "index": "not_analyzed"
          },
          "id": {
            "type": "string",
            "index": "not_analyzed"
          },
          "tenantId": {
            "type": "string",
            "index": "not_analyzed"
          }
        }
      }
    }
  }
}

JAVA CLIENT

(You can ignore the "Observable" related stuff for now)

public Observable<Boolean> createAlias(String tenantId) {

        FilterBuilder filter = FilterBuilders.termFilter("tenantId", tenantId);
        ListenableActionFuture<IndicesAliasesResponse> response = client.admin().indices().prepareAliases().addAlias("mdm-master", tenantId, filter).execute();
        return Observable.from(response)
                .map((IndicesAliasesResponse apiResponse) -> {
                    return apiResponse.isAcknowledged();
                });
    }

anderso anderso · Accepted Answer · 2015-03-11T22:33:31

I'm the guy who posted that latest issue on ES Github Failure to create Filtered Alias on empty index with template mappings #10038. The quickest workaround I've found for now (except downgrading to 1.3, where this issue does not exist), is to index a document with the field before creating the alias.

If you have one index with many tenants, you should only need to index a document with the required field once when you create the index, and then you should be able to create the alias.

If you try the reproduction case I posted in the GitHub issue, but before creating the alias, run the following:

curl -XPOST  'http://localhost:9200/repro/dummytype/1' -d '{
   "testfield": "dummyvalue"
}'

Then you should be able to add a filtered alias on the field testfield.

Edit - Answer to first comment: I think that it's an oversight when you use mapping in templates. A template is applied to matching index when the index is created. I think the issue here is that the generic mapping part of the template isn't actually applied until it gets a document indexed. This behaviour can be observed if you change my template in the issue to the following:

curl -XPUT 'http://localhost:9200/_template/repro' -d '{
  "template": "repro",
  "settings": {
    "index.number_of_shards": 1,
    "index.number_of_replicas": 0
  },
  "mappings": {
    "dummytype": {
      "properties": {
        "testfield": {
          "type": "string",
          "index": "not_analyzed"
        }
      }
    }
  }
}'

Then you're able to create the index and add the filtered alias without indexing any documents.

As I said, I think this is a bug in the template application in ES.

Elastic Search filtered aliases with unmapped fields

1 Answers