Elasticsearch: exception caught on transport layer

3
votes

After upgrading Elasticsearch from version 1.6 to 2.1 I see an error in my Java application.

The application used to work with Java before, but now the connection gets terminated immediately and a similar error (org.elasticsearch.client.transport.NoNodeAvailableException) is thrown in the application as result.

It all happens during first connection attempt, which points to a clash between the versions, but may as well be something else.

What can it be specifically, is there any way to fix it other than changing the versions?

elasticsearch.log:

[2015-12-08 17:42:54,035][WARN ][transport.netty          ] [Lorna Dane] exception caught on transport layer [[id: 0x9f75ad33, /192.168.0.208:21248 => /192.168.0.140:9300]], closing connection
java.lang.IllegalStateException: Message not fully read (request) for requestId [0], action [cluster/nodes/info], readerIndex [39] vs expected [57]; resetting
        at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:120)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Java console:

Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:298)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:214)
    at org.elasticsearch.client.transport.support.InternalTransportClusterAdminClient.execute(InternalTransportClusterAdminClient.java:85)
    at org.elasticsearch.client.support.AbstractClusterAdminClient.state(AbstractClusterAdminClient.java:138)
    at org.elasticsearch.action.admin.cluster.state.ClusterStateRequestBuilder.doExecute(ClusterStateRequestBuilder.java:94)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
2
javaparsingelasticsearchjbosstransport
What does your cluster consist of node-wise? Did you upgrade all ES nodes and reindex your data? - Chris
It's just a single instance that I'm connecting to. It was upgraded and the data were not moved yet, that's what the application is for. - Peter G.
Did you remove your prev jar files in your IDE in favor of the new Elastic jar files to connect properly to new ver? Did you set your cluster name in new ver? - Chris
No, I didn't. It would make sense to move them. - Peter G.

2 Answers

2
votes

Make sure to keep the elasticsearch client library jar in sync with the version of your cluster.

For example when using Maven:

 <dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>2.1.0</version>
 </dependency>
0
votes

This happens for any >2.x Elasticsearch instance anytime there is still a 1.x machine looking for clusters (it sends bad protocol information) on the same network. Technically you can ignore it (it's just letting you know that the 1.x machine tried to connect but did an invalid connection), however probably your logs will fill up etc. if you leave it alone too long.

Best solution is to isolate your old 1.x nodes from the newer nodes until you're done upgrading.

see: https://github.com/elastic/elasticsearch/issues/14400