How to add lowercase normalizer to Elasticsearch mapping object in a particular field?

Question

I have a schema published into an elastic, and I want to know if is possible add a lowercase normalizer when the index is published in order to update that field and avoid define a new schema object with all old fields and the new one.

Currently I am triying to update that field with this command:

await this.getClient().indices.putSettings({index: indexName, body:{ 
  softwarePublisher: {
    type: "text",
    fields: {
      ngram: {
        type: "text",
        analyzer: "software_analyzer",
        search_analyzer: "software_search_analyzer",
      },
      raw: {
        type: "keyword",
      },
      rawl: {
        type: "keyword",
        normalizer: "lowercase_normalizer",
      },
    },
  },
}});

But I got the following error:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.ngram.analyzer] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"}],"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.ngram.analyzer] please check that any required plugins are installed, or check the breaking changes documentation for removed settings","suppressed":[{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.ngram.search_analyzer] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.ngram.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.raw.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.rawl.normalizer] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.fields.rawl.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},{"type":"illegal_argument_exception","reason":"unknown setting [index.softwarePublisher.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"}]},"status":400}

I tried to with putMappings but anyway I got an error.

EDIT

Elastic version

curl -XGET 'http://localhost:9200'
{
  "name" : "bf9cd8c528fc",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "MzMfZzJaQwia1A5U5SFzNg",
  "version" : {
    "number" : "7.1.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "7a013de",
    "build_date" : "2019-05-23T14:04:00.380842Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

EDIT 2

Error using putMapping

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [publisher : {type=text, fields={rawl={normalizer=lowercase_normalizer, type=keyword}, raw={type=keyword}, ngram={search_analyzer=software_search_analyzer, analyzer=software_analyzer, type=text}}}]"}],"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [publisher : {type=text, fields={rawl={normalizer=lowercase_normalizer, type=keyword}, raw={type=keyword}, ngram={search_analyzer=software_search_analyzer, analyzer=software_analyzer, type=text}}}]"},"status":400}

Analyzer

export const softwareNameAnalyzer = {
  filter: {
    ngram_custom: {
      type: "edge_ngram",
      min_gram: "1",
      max_gram: "15",
    },
  },
  tokenizer: {
    software_tokenizer,
  },
  normalizer: {
    lowercase_normalizer: {
      filter: ["lowercase"],
    },
  },
  analyzer: {
    software_search_analyzer: defaultAnalyzer,
    software_analyzer: softwareAnalyzer,
    default: defaultAnalyzer,
    default_search: defaultAnalyzer,
  },
};

export const softwareAnalyzer = {
  type: "custom",
  tokenizer: "software_tokenizer",
  char_filter: ["html_strip"],
  filter: ["lowercase", "ngram_custom"],
};

Any idea?

Thanks?

You should use putMappings() not putSettings() since you're defining fields. What error do you get with putMappings()? — Val

Val Val · Accepted Answer · 2021-10-13T12:17:23

First off, you should use putMappings() not putSettings() since you're defining fields.

Then you must invoke putMappings() as follows (you're missing the properties section):

await this.getClient().indices.putMappings({index: indexName, body:{ 
  properties: {                            <<----- add this
      softwarePublisher: {
         ...
      }
  }
}});

Note: You need to make sure to invoke putSettings() before putMappings so that your custom analyzers are installed first, otherwise the putMappings() call will complain that your custom analyzers don't exist.

How to add lowercase normalizer to Elasticsearch mapping object in a particular field?

1 Answers