Re-Index Elasticsearch, ignore fields not in mapping

Question

Trying to test out re-index API in elasticsearch and running into issues where existing data contains fields not present in the new index's strict mapping. Is there a way to tell elasticsearch to simply ignore those fields and carry on?

Edit: To clarify, by ignore I meant not to include those fields during the re-index process.

can you adjust the mapping settings before starting the reindex? or you don't have access to it? — Andrey Borisko
@AndreyBorisko I can but I want the fields ignored, or i guess removed, during the re-index. Is that even possible? — MarkII
it's possible. might look dirty though. depending on how many properties you need to remove. — Andrey Borisko

Andrey Borisko Andrey Borisko · Accepted Answer · 2019-09-11T21:49:05

If you have access to the index settings before running reindex you can just do:

PUT test/_mapping
{
    "dynamic": "false"
}

then change it back to strict once reindexing is done.

UPDATE based on your comment

POST _reindex
{
  "source": {
    "index": "src"
  },
  "dest": {
    "index": "dst"
  },
  "script": {
    "lang": "painless",
    "source": """
    ctx['_source'].remove('email');
    ctx['_source'].remove('username');
    ctx['_source'].remove('name');
    // removing from nested:
    for(item in ctx['_source'].Groups){
        item.remove('GroupName');
        item.remove('IsActive');
    }
    """
  }
}

Re-Index Elasticsearch, ignore fields not in mapping

1 Answers