InvalidQueryException: Consistency level LOCAL_ONE is not supported for this operation. Supported consistency levels are: LOCAL_QUORUM

Question

import org.apache.spark._
import org.apache.spark.SparkContext._
import com.datastax.spark.connector._
import com.datastax.spark.connector.cql.CassandraConnector    
val conf = new SparkConf()
          .setMaster("local[*]")
          .setAppName("XXXX")
          .set("spark.cassandra.connection.host" ,"cassandra.us-east-2.amazonaws.com")
          .set("spark.cassandra.connection.port", "9142")
          .set("spark.cassandra.auth.username", "XXXXX")
          .set("spark.cassandra.auth.password", "XXXXX")
          .set("spark.cassandra.connection.ssl.enabled", "true")
          .set("spark.cassandra.connection.ssl.trustStore.path", "/home/nihad/.cassandra/cassandra_truststore.jks")
          .set("spark.cassandra.connection.ssl.trustStore.password", "XXXXX")
          .set("spark.cassandra.output.consistency.level", "LOCAL_QUORUM")
    
    val connector = CassandraConnector(conf)
           val session = connector.openSession()
               sesssion.execute("""INSERT INTO "covid19".delta_by_states (state_code, state_value, date ) VALUES ('kl', 5, '2020-03-03');""")
session.close()

i amn trying to write data to AWS Cassandra Keyspace using Spark App set in my local system. Problem is when i execute above code, I get Exception like below:

"com.datastax.oss.driver.api.core.servererrors.InvalidQueryException: Consistency level LOCAL_ONE is not supported for this operation. Supported consistency levels are: LOCAL_QUORUM"

As you can see from the above code I have already set cassandra.output.consistency.level as LOCAL_QUORUM in Spark Conf. Also I am using datastax cassandra driver.

But when I read data from AWS Cassandra, it works fine. Also I tried same INSERT command in AWS Keyspace cqlsh. It is working fine there too. So Query is valid.

Can someone help me how to set consistency via datastax.CassandraConnector?

Possible duplicate? Does this help? stackoverflow.com/questions/60943631/… — Aaron
@Aaron I tried that already It does not help since datastax java driver does not support a second argument for setting consistency. Check the doc [here] (docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/…). The second argument can only be used for binding query values to a prepared query statement . Please correct me if I am wrong. — Nihad TP
So the document you referenced was written in 2015. I guess it's relevant if 2.0 is the version of the driver you're using, but I seriously hope you're using one newer than that. — Aaron
@Aaron Managed to crack this with application.conf. Find my answer below — Nihad TP

Nihad TP Nihad TP · Accepted Answer · 2020-08-23T13:37:16

Cracked it.

Instead of setting cassandra consistency via spark config. I created an application.conf file in src/main/resources directory.

    datastax-java-driver {

        basic.contact-points = [ "cassandra.us-east-2.amazonaws.com:9142"]
    advanced.auth-provider{
        class = PlainTextAuthProvider
        username = "serviceUserName"
        password = "servicePassword"
    }
    basic.load-balancing-policy {
        local-datacenter = "us-east-2"
    }

    advanced.ssl-engine-factory {
        class = DefaultSslEngineFactory
        truststore-path = "yourPath/.cassandra/cassandra_truststore.jks"
        truststore-password = "trustorePassword"
      }

    basic.request.consistency = LOCAL_QUORUM 
    basic.request.timeout = 5 seconds 
    
}

and created cassandra session like below

import com.datastax.oss.driver.api.core.config.DriverConfigLoader
import com.datastax.oss.driver.api.core.CqlSession

val loader = DriverConfigLoader.fromClassPath("application.conf")
val session = CqlSession.builder().withConfigLoader(loader).build()
sesssion.execute("""INSERT INTO "covid19".delta_by_states (state_code, state_value, date ) VALUES ('kl', 5, '2020-03-03');""")

It finally worked. No need to mess with spark config Doc for Driver Config https://docs.datastax.com/en/drivers/java/4.0/com/datastax/oss/driver/api/core/config/DriverConfigLoader.html#fromClasspath-java.lang.String-

datastax configuration doc https://docs.datastax.com/en/developer/java-driver/4.6/manual/core/configuration/reference/

InvalidQueryException: Consistency level LOCAL_ONE is not supported for this operation. Supported consistency levels are: LOCAL_QUORUM

1 Answers