How to generate a UUID in Elasticsearch reindex painless script

2
votes

I am trying to use the reindex api to create a copy of a set of documents. One of the fields (uuid) of the document is a UUID. I need the copied documents to have new UUIDs for the uuid field.

According to [1] and [2] the method java.util.UUID.randomUUID() is not whitelisted for use in painless scripts.

Questions:

1) How can I generate a UUID in painless?

2) Why is UUID.randomUUID() considered an unsafe operation? Or is it just an oversight that it is not whitelisted?

3) How can I whitelist UUID.randomUUID() in the "reindex" context? I tried to build my own elasticsearch painless extension/plugin to do this based on the example in [3]. The problem is it only works for the "SearchScript" context. There does not seem to be an equivalent "ReindexContext".

Here is a simplified version of what I am trying:

curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "product1"
  },
  "dest": {
    "index": "product2"
  },
  "script": {
    "source": "ctx._source.uuid = java.util.UUID.randomUUID().toString()",
    "lang": "painless"
  }
}
'

Which produces the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "compile error",
        "script_stack" : [
          "... rce.uuid = java.util.UUID.randomUUID().toString()",
          "                             ^---- HERE"
        ],
        "script" : "ctx._source.uuid = java.util.UUID.randomUUID().toString()",
        "lang" : "painless"
      }
    ],
    "type" : "script_exception",
    "reason" : "compile error",
    "script_stack" : [
      "... rce.uuid = java.util.UUID.randomUUID().toString()",
      "                             ^---- HERE"
    ],
    "script" : "ctx._source.uuid = java.util.UUID.randomUUID().toString()",
    "lang" : "painless",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "method [java.util.UUID, randomUUID/0] not found"
    }
  },
  "status" : 500
}

I know my approach is valid and that the above is a painless whitelisting issue because when I try a different method (fromString()) I get no errors:

curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "product1"
  },
  "dest": {
    "index": "product2"
  },
  "script": {
    "source": "ctx._source.uuid = java.util.UUID.fromString(\u0027ad139caa-5b54-4179-b812-5015daecad1e\u0027).toString()",
    "lang": "painless"
  }
}
'

References:

[1] - https://discuss.elastic.co/t/generate-a-uuid-using-randomuuid-in-painless/144354/3

[2] - https://www.elastic.co/guide/en/elasticsearch/painless/6.6/painless-api-reference.html

[3] - https://github.com/elastic/elasticsearch/tree/v6.6.0/plugins/examples/painless-whitelist

Other Notes:

3
elasticsearchelasticsearch-painless
May be define your own custom implementation/method in the script to generate uuid's as per your requiremen and use that method. Basically, work around the dependency - Polynomial Proton
My feature request to resolve this issue has been accepted and implemented here: github.com/elastic/elasticsearch/issues/39080 - Oliver Henlich

3 Answers

1
votes

My feature request to resolve this issue has been accepted and implemented here: https://github.com/elastic/elasticsearch/issues/39080

0
votes
You can simply do the following,

curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "product1"`enter code here`
  },
  "dest": {
    "index": "product2"
  },
  "script": {
    "source": "ctx._id=ctx._id+1",
    "lang": "painless"
  }
}
'
ctx._id = will always give you a new id and plus 1 will generate the new one.

this is just a solution to get unique id by adding suffix

0
votes

The most straightforward approach is to use a self-written function, here is mine. Of course it is just a workaround but it should help in most cases.

String generateUUID(boolean addDashes, boolean upperCase) {
    def chars = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'];

    def dashIndices = [7, 11, 15, 19];

    def sb = new StringBuilder();

    def r = new Random();

    for (def pos = 0; pos < 32; pos += 1) {
       sb.append(chars[r.nextInt(16)]);

       if (addDashes && dashIndices.contains(pos)) {
         sb.append('-');
       }
    }

    def result = sb.toString();

    return upperCase ? result.toUpperCase() : result;
}

Copy it to your script and below you will be able to get UUID by calling, for example, generateUUID(true, false), if you need UUID with dashes and in lower case.