How to fix "java.lang.ArrayIndexOutOfBoundsException" exception while starting elastic search in 6.2 version

Question

Elastic search server doesn't start on a new node. It fails with the following error :

[2019-06-27T00:16:01,471][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-10] fatal error in thread [main], exiting
java.lang.ExceptionInInitializerError: null
    at java.lang.Class.forName0(Native Method) ~[?:1.8.0_212]
    at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_212]
    at org.elasticsearch.painless.Definition.addStruct(Definition.java:753) ~[?:?]
    at org.elasticsearch.painless.Definition.<init>(Definition.java:566) ~[?:?]
    at org.elasticsearch.painless.PainlessScriptEngine.<init>(PainlessScriptEngine.java:106) ~[?:?]
    at org.elasticsearch.painless.PainlessPlugin.getScriptEngine(PainlessPlugin.java:59) ~[?:?]
    at org.elasticsearch.script.ScriptModule.<init>(ScriptModule.java:69) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:327) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.2.jar:6.2.2]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
    at java.time.chrono.JapaneseEra.<clinit>(JapaneseEra.java:179) ~[?:1.8.0_212]
    ... 19 more

I have a 5-node cluster running in production already in GCP. Since the load has increased, I tried to add few more nodes to that cluster. To create new nodes, I used "Create similar" option provided by GCP. I updated all configurations like node name and deleted the /var/lib/elasticsearch/nodes folder and tried to start the ES server on the new node. But it always fails with the error mentioned above.

This node uses OpenJDK 1.8. I enabled trace log for root logger, but couldn't identify what's wrong with the new node.

Please help in identifying what's the root cause of this problem.

this looks awfully similar: discuss.elastic.co/t/error-at-java-time-chrono-japaneseera/… — Val
Thanks Val; I tried suggestion mentioned in that post and server started fine. However, it is strange that few instances showed this problem while other instances created exactly using the same steps did not show any errors. — Shobhana Sriram
Good. Are you certain that each instance has the exact same OpenJDK version (+build number)? — Val
Yes, since JDK was installed BEFORE taking the image of the original instance. All new instances were created using the same image! — Shobhana Sriram

Swapnil G Thaware Swapnil G Thaware · Accepted Answer · 2019-12-17T13:57:11

Theory explanation:

public class ArrayIndexOutOfBoundsException extends IndexOutOfBoundsException

Thrown to indicate that an array has been accessed with an illegal index. The index is either negative or greater than or equal to the size of the array.

How to avoid:

Always remember that array is zero-based index, the first element is at 0th index and the last element is at length - 1 index. So accessing the first element will give you the java.lang.ArrayIndexOutOfBoundsException : 0 error in Java.

In your case it is a Caused by: java.lang.ArrayIndexOutOfBoundsException: 4

You should always pay to one-off errors while looping over an array in Java. The programmer often makes mistakes which result in either missing first or last element of the array by messing 1st element or finishing just before the last element by incorrectly using the <, >,> >= or <= operator in for loops.

Give special attention to the start and end condition of the loop.
Put some if-else blocks in code.

Here I have reproduced your error:

How to fix "java.lang.ArrayIndexOutOfBoundsException" exception while starting elastic search in 6.2 version

1 Answers