Defining index templates on ElasticSearch with .NET client

Question

I have ElasticSearch running in a Docker cluster, I use it to search through a ASP.NET core API that queries ES. I would like to define the index mappings for ES at cluster creation, running a job in my docker-compose that uses a .NET console app to put the requests on ES to define the mappings.

I have seen that using templates should do the job for me, but I can't find any examples for best practices in NEST or ElasticSearch-NET so I thought I'd ask here.

My idea would be to simply define a template that is applied to any new index that is created. Then I run an external job that seeds ES and any new index will have the same mapping applied to it whenever a new document needs to go in a new index. I also want to define an alias that is common to all the new indexes.

The mappings I want to define are as below:

{
    "settings": {
      "analysis": {
        "filter": {
          "autocomplete_filter": {
            "type": "edge_ngram",
            "min_gram": 1,
            "max_gram": 20
          }
        },
        "analyzer": {
          "autocomplete": {
            "type": "custom",
            "tokenizer": "standard",
            "filter": [
              "lowercase",
              "autocomplete_filter"
            ]
          }
        }
      }
    },
    "mappings": {
      "dynamic": false,
      "properties": {
        "Name": {
          "type": "keyword"
        },
        "Cusip": {
          "type": "keyword"
        },
        "ISIN": {
          "type": "keyword"
        },
        "Ticker": {
          "type": "keyword"
        },
        "suggest": {
          "type": "completion",
          "analyzer": "autocomplete",
          "search_analyzer": "standard"
        }
      }
    }
}

And for the alias I need to use something like this:

{
          "actions" : [
              { "add" : { "index" : "{index}", "alias" : "product" } }
          ]
      }

Questions:

Is using a template the right way to do this?
How do I package it together into a template?
How do I ensure that all new indexes have these settings and that it doesn't apply to like the metrics or other default indexes created by ES?
Do templates also cover preferences on how to return the _source after a search? E.g. if I want to consistently exclude a field that is added for the autosuggest feature but that I don't want returned on normal queries?

Thanks in advance for your help

Simon

szan szan · Accepted Answer · 2020-09-21T00:45:44

In the end is solved this using the LowLevel Elasticsearch-net client. I have a config file that contains the JSON request for the template:

  "index_patterns": [
    "*"
  ],
  "settings": {
    "analysis": {
      "filter": {
        "autocomplete_filter": {
          "type": "edge_ngram",
          "min_gram": 1,
          "max_gram": 20
        }
      },
      "analyzer": {
        "autocomplete": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "autocomplete_filter"
          ]
        }
      }
    }
  },
  "mappings": {
    "dynamic": false,
    "properties": {
      "Name": {
        "type": "keyword"
      },
      "field1": {
        "type": "keyword"
      },
      "field2": {
        "type": "keyword"
      },
      "field3": {
        "type": "keyword"
      },
      "suggest": {
        "type": "completion",
        "analyzer": "autocomplete",
        "search_analyzer": "standard"
      }
    }
  },
  "aliases" : {
    "product" : {}
  }
}

And I send the request for the template mapping with this:

// Send a PUT request containing the template
var postMapping =
        _client.LowLevel.DoRequest<StringResponse>(HttpMethod.PUT, "_template/product_template", PostData.String(template.ToString()));

Defining index templates on ElasticSearch with .NET client

1 Answers