Can't connect to ElasticSearch server using Java API

22
votes

I am trying to connect to an ElasticSearch server using the Java API. I using elasticsearch service to start/stop and elasticsearch head for visualising the cluster. The cluster/node is active, the REST API works fine via curl on 9200. I have read pretty much every posts about this topic, yet I can't get it working, below are my details:

I have tried all these:

  1. Using localhost,devhost1,127.0.0.1,or the actual IP on the network in TransportClient
  2. Uncommenting the IP properties in config.yaml and putting 127.0.0.1 (server)
  3. Checking if port 9300 is available, it is.

...9200/_cluster/nodes

{
  "ok": true,
  "cluster_name": "test",
  "nodes": {
    "NLVBbJpJTZWefeI2kQt3Tg": {
      "name": "inventory_management",
      "transport_address": "inet[/127.0.0.1:9300]",
      "hostname": "devhost1",
      "version": "0.90.5",
      "http_address": "inet[/127.0.0.1:9200]"
    }
  }
}

...9200/_cluster/health?pretty=true

{
  "cluster_name" : "test",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

Java Code:

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "test").put("node.name", "inventory_management").build();
Client client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("localhost", 9300));
IndexResponse response = client.prepareIndex("twitter", "tweet", "1").setSource(json).execute().actionGet();

Exception:

org.elasticsearch.transport.NodeDisconnectedException: [][inet[localhost/127.0.0.1:9300]][/cluster/nodes/info]
org.elasticsearch.client.transport.NoNodeAvailableException: No node available
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:138)
    at org.elasticsearch.client.transport.support.InternalTransportClient.index(InternalTransportClient.java:124)
    at org.elasticsearch.client.transport.TransportClient.index(TransportClient.java:242)
    at org.elasticsearch.client.action.index.IndexRequestBuilder.doExecute(IndexRequestBuilder.java:219)
    at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:52)
    at org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:47)

Log from Client (Java API):

[2013-10-24 16:37:15,783][DEBUG][threadpool.cached        ] [Aragorn] Initializing cached thread pool with keep_alive[1m], scheduled_size[20]
[2013-10-24 16:37:15,809][DEBUG][client.transport         ] [Aragorn] node_sampler_interval[1s]
[2013-10-24 16:37:15,820][DEBUG][netty.channel.socket.nio.NioProviderMetadata] Using the autodetected NIO constraint level: 0
[2013-10-24 16:37:15,872][DEBUG][transport.netty          ] [Aragorn] Connected to node [[#temp#-1][inet[localhost/127.0.0.1:9300]]]
[2013-10-24 16:37:15,892][DEBUG][transport.netty          ] [Aragorn] Disconnected from [[#temp#-1][inet[localhost/127.0.0.1:9300]]]
[2013-10-24 16:37:15,894][DEBUG][client.transport         ] [Aragorn] Failed to get node info from [#temp#-1][inet[localhost/127.0.0.1:9300]], removed from nodes list
org.elasticsearch.transport.NodeDisconnectedException: [][inet[localhost/127.0.0.1:9300]][/cluster/nodes/info]

+++++++++++++++++++++++

Tried with Unicast same error in the Java API.

discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: "localhost" # Tried with devhost1 127.0.0.1 as well and ["localhost"] too

This is the server log: 

[2013-10-25 13:49:58,379][INFO ][node                     ] [inventory_management] version[0.90.5], pid[2426], build[c8714e8/2013-09-17T12:50:20Z]
[2013-10-25 13:49:58,379][INFO ][node                     ] [inventory_management] initializing ...
[2013-10-25 13:49:58,382][INFO ][plugins                  ] [inventory_management] loaded [], sites []
[2013-10-25 13:49:59,853][INFO ][node                     ] [inventory_management] initialized
[2013-10-25 13:49:59,853][INFO ][node                     ] [inventory_management] starting ...
[2013-10-25 13:49:59,937][INFO ][transport                ] [inventory_management] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.50:9300]}
[2013-10-25 13:50:02,960][INFO ][cluster.service          ] [inventory_management] new_master [inventory_management][2-uG1xVNSSiLo5RVRrjbGg][inet[/192.168.1.50:9300]], reason: zen-disco-join (elected_as_master)
[2013-10-25 13:50:02,972][INFO ][discovery                ] [inventory_management] test/2-uG1xVNSSiLo5RVRrjbGg
[2013-10-25 13:50:02,989][INFO ][http                     ] [inventory_management] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.50:9200]}
[2013-10-25 13:50:02,989][INFO ][node                     ] [inventory_management] started
[2013-10-25 13:50:03,024][INFO ][gateway                  ] [inventory_management] recovered [0] indices into cluster_state
[2013-10-25 13:50:09,399][WARN ][transport.netty          ] [inventory_management] exception caught on transport layer [[id: 0x02649775, /127.0.0.1:50028 => /127.0.0.1:9300]], closing connection
java.io.StreamCorruptedException: invalid internal transport message format
    at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:27)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)
[2013-10-25 13:50:09,403][WARN ][transport.netty          ] [inventory_management] exception caught on transport layer [[id: 0x02649775, /127.0.0.1:50028 :> /127.0.0.1:9300]], closing connection
java.io.StreamCorruptedException: invalid internal transport message format
    at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:27)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:482)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:365)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:102)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireChannelDisconnected(Channels.java:396)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:361)
    at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:81)
    at org.elasticsearch.common.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:574)
    at org.elasticsearch.common.netty.channel.Channels.close(Channels.java:812)
    at org.elasticsearch.common.netty.channel.AbstractChannel.close(AbstractChannel.java:197)
    at org.elasticsearch.transport.netty.NettyTransport.exceptionCaught(NettyTransport.java:501)
    at org.elasticsearch.transport.netty.MessageChannelHandler.exceptionCaught(MessageChannelHandler.java:228)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377)
    at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
    at org.elasticsearch.common.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:566)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)

+++++++++++++++++++++++
+++++++++++++++++++++++

Java version:

java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.12) (7u25-2.3.12-4ubuntu3)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

+++++++++++++++++++++++
+++++++++++++++++++++++
+++++++++++++++++++++++

Issue has been solved as per below. Make sure your client and server versions are in sync!

4
javaelasticsearch
I think the problem might be related to having multiple network devices on your machine. You are running both client and server on your local machine right? Otherwise it's a discovery problem using multicast...any firewall? What operating system? – javanna
Hi, yes that's right both is running on the same machine (LINUX/Kubuntu). No firewall. – k9m
...Only got one network device. – k9m
have you tried using the more explicit unicast cluster discovery vs. multicast auto discovery? – mconlin
Just so you know, the initial ping on discovery use the 9300 port, but udp instead of tcp. Again, usually this kind of problems are due to firewalls. I would definitely give a try to unicast instead of multicast discovery as @mconlin said. – javanna

4 Answers

25
votes

Just so this question is marked answered for others who may experience the same issue (the answer is actually at the end of the question).

I ran into the same issue and it turned out to be a discrepancy between the version of the JAR used by the Java client and the version the server was running. Your best bet is to ensure an exact match, then the instructions given here just work w/o any tweaking.

Another thing to check is that your Java client is using the correct port, which is not the one used by http clients. The correct port defaults to 9300 rather than 9200, which is used by the latter.

11
votes

In case anybody else stumbles on this thread, I found out another reason you could receive this error is because the cluster_name has been changed to something other than the default.

To see your current cluser_name:

curl -XGET localhost:9200/_cluster/nodes?pretty=true

Use code similar to what OP has above:

Settings settings = ImmutableSettings.settingsBuilder()
    .put("cluster.name", "test").build();
Client client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("localhost", 9300));

Source: http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html

1
votes

First, version is really important, Java API is strong related with the version you are using,

This solution works in elasticSearch v 5.4.1 with java 8

make sure that you are getting all dependencies, and the child dependencies (not only elasticSearch jar), use maven, ivy etc for getting the dependencies tree.

in ivy for example : 

 <!-- https://mvnrepository.com/artifact/org.elasticsearch.client/transport -->
  <dependency org="org.elasticsearch.client" name="transport" rev="5.4.1"  conf="default"/>

//--------------------- then in a class you can call: 

 TransportClient client;

    client = new PreBuiltTransportClient(Settings.EMPTY).addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));
0
votes

This is for users who are using spring-boot spring-data elasticsearch. Please ensure that the version of your client is the same as the server. 

<library name="Gradle: org.elasticsearch:elasticsearch:1.5.2">
  <CLASSES>
    <root url="jar://$USER_HOME$/.gradle/caches/modules-2/files-2.1/org.elasticsearch/elasticsearch/1.5.2/47aafc6bf8f23ed8dcbf6a1db174fb0b8e44a8db/elasticsearch-1.5.2.jar!/" />
  </CLASSES>
  <JAVADOC />
  <SOURCES>
    <root url="jar://$USER_HOME$/.gradle/caches/modules-2/files-2.1/org.elasticsearch/elasticsearch/1.5.2/d1e0b7b758ce7bd5d6e3757896054b09f28f372d/elasticsearch-1.5.2-sources.jar!/" />
  </SOURCES>
</library>