NoHostAvailableException with 1000 concurrent request to Cassandra with DataStax Java Driver

Question

The setup:

-2-node Cassandra 2.0.7.31 cluster
-replicas=1
-With default configuration
-Using DataStax java driver 1.0

Activity Simple insert query using QueyBuilder class

Result

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.181.13.239 ([/10.181.13.239] Unexpected exception triggered))
            at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:64)
            at com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException(ResultSetFuture.java:214)
            at com.datastax.driver.core.ResultSetFuture.getUninterruptibly(ResultSetFuture.java:169)
            at com.jpmc.es.rtm.storage.impl.EventExtract.main(EventExtract.java:36)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:601)
            at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.181.13.239 ([/10.181.13.239] Unexpected exception triggered))
            at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:98)
            at com.datastax.driver.core.RequestHandler$1.run(RequestHandler.java:165)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

are you able to connect to the cassandra server using cassandra-cli ? — Vishal John
yes. I am getting around 85 TPS for upsert for 1000 concurrent hit . But I have more expectation from Cassandra regarding TPS. I am confused is that the datastax driver which is slowing down the whole thing? — Sarkar

Sarkar Sarkar · Accepted Answer · 2014-07-26T11:52:13

Problem was that at my end I have created one thread one connection model. Which is quite synchronous. But Datstax driver works asynchronously(its using netty for that i guess) and taking multiple request on single connection. so in my case I one session for each request and guess what? I have left with pool of open connection b/w Driver and Cassandra. Server got choked so does the Driver. Problem was solved simply by letting Driver manage its pool.

Datastax java driver by default maintain minimum amount of connection for handling Certain amount of simultaneous Request.
Spec says that driver have a limitation of handling at the most 128 request per connection.so When we driver find that more than 128 request is coming Then it only open connection. So Driver is beautifully managing connection pool.[ http://www.datastax.com/drivers/java/2.0/com/datastax/driver/core/PoolingOptions.html] This link was helpful

NoHostAvailableException with 1000 concurrent request to Cassandra with DataStax Java Driver

2 Answers