Disable dynamic mapping creation for only specific indexes on elasticsearch?

Question

I'm trying to disable dynamic mapping creation for only specific indexes, not for all. For some reason I can't put default mapping with 'dynamic' : 'false'. So, here left two options as I can see:

1. specify property 'index.mapper.dynamic' in file elasticsearch.yml.
2. put 'index.mapper.dynamic' at index creation time, as described here https://www.elastic.co/guide/en/kibana/current/setup.html#kibana-dynamic-mapping

First option may only accept values: true, false and strict. So there is no way to specify subset of specific indexes (like we do by pattern with property 'action.auto_create_index' https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation).

Second option just not works. I've created index

POST http://localhost:9200/test_idx/
{
    "settings" : {
        "mapper" : {
            "dynamic" : false
        }
    },
    "mappings" : {
        "test_type" : {
            "properties" : {
                "field1" : {
                    "type" : "string"
                }
            }
        }
    }
}

Then checked index settings:

GET http://localhost:9200/test_idx/_settings    
{
    "test_idx" : {
        "settings" : {
            "index" : {
                "mapper" : {
                    "dynamic" : "false"
                },
                "creation_date" : "1445440252221",
                "number_of_shards" : "1",
                "number_of_replicas" : "0",
                "version" : {
                    "created" : "1050299"
                },
                "uuid" : "5QSYSYoORNqCXtdYn51XfA"
            }
        }
    }
}

and mapping:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

so far so good, let's index document with undeclared field:

POST http://localhost:9200/test_idx/test_type/1
{
    "field1" : "it's ok, field must be in mapping and in source",
    "somefield" : "but this field must be in source only, not in mapping"
}

Then I've checked mapping again:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    },
                    "somefield" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

As you can see, mapping is extended regardless of index setting "dynamic" : false. I've also tried to create index exactly as described in doc

PUT http://localhost:9200/test_idx
{
    "index.mapper.dynamic": false
}

but got the same behavior.

Maybe I've missed something?

Thanks a lot in advance!

Answer 1

You're almost there: the value needs to be set to strict. And the correct usage is the following:

PUT /test_idx
{
  "mappings": {
    "test_type": {
      "dynamic":"strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

And pushing this a bit further, if you want to forbid the creation even of new types, not only fields in that index, use this:

PUT /test_idx
{
  "mappings": {
    "_default_": {
      "dynamic": "strict"
    },
    "test_type": {
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

Without _default_ template:

PUT /test_idx
{
  "settings": {
    "index.mapper.dynamic": false
  },
  "mappings": {
    "test_type": {
      "dynamic": "strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

Answer 2

You must know about that the below part just mean that ES could'nt create a type dynamically.

"mapper" : {
        "dynamic" : false
    }

You should configure ES like this:

PUT http://localhost:9200/test_idx/_mapping/test_type
{
  "dynamic":"strict"
}

Then you cant't index other field that without mapping any more ,and get an error as follow:

mapping set to strict, dynamic introduction of [hatae] within [data] is not allowed

If you wanna store the data,but make the field can't be index,you could take the setting like this:

PUT http://localhost:9200/test_idx/_mapping/test_type
{
  "dynamic":false
}

Hope these can help the people with the same issue :).

Answer 3

The answer is in the doc (7x.): https://www.elastic.co/guide/en/elasticsearch/reference/7.x/dynamic.html

The dynamic setting controls whether new fields can be added dynamically or not. It accepts three settings:

true

Newly detected fields are added to the mapping. (default)

false

Newly detected fields are ignored. These fields will not be indexed so will not be searchable but will still appear in the _source field of returned hits. These fields will not be added to the mapping, new fields must be added explicitly.

strict

If new fields are detected, an exception is thrown and the document is rejected. New fields must be explicitly added to the mapping.

PUT my_index
{
  "mappings": {
    "dynamic": "strict", 
    "properties": {
      "user": { 
        "properties": {
          "name": {
            "type": "text"
          },
          "social_networks": { 
            "dynamic": true,
            "properties": {}
          }
        }
      }
    }
  }
}

Answer 4

You cannot disable dynamic mapping in ES 7 anymore, what you can do if you have completely unstructured data is to disable completely the mapping for the index like this:

curl -X PUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "enabled": false 
  }
}
'

if you are using python you can do this:

from elasticsearch import Elasticsearch

# Connect to the elastic cluster
es=Elasticsearch([{'host':'localhost','port':9200}])

request_body = {
        "mappings": {
                 "enabled": False
    }
}
es.indices.create(index = 'my_index', body = request_body)

Answer 5

For ES 7 if you want to update an existing index:

PUT customers/_mapping
{
  "dynamic": "strict"
}