3
votes

We are doing hourly aggregations using Spark SQL and Cassandra on huge data. We have developed a Java client which runs every hour to do the aggregations using Spark SQL. For historic loads, When we running this program for 10 days (240 Hours) of data, after around 100 hours processed, Cassandra is failing with below error:

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)

what's causing Cassandra to fail ?

1
what version of the spark connector / datastax java driver are you using? - Andy Tolbert
We are using spark-cassandra connector 1.2.0-rc3 jar. Spark version is 1.2.1 and cassandra version is 2.0.13 - Ramana

1 Answers

2
votes

This is resolved after configuring higher values for "spark.cassandra.read.timeout_ms" and "spark.cassandra.connection.timeout_ms"