How to read specific columns from Cassandra table using Datastax spark-cassandra-connector?

Question

I am using spark-cassandra-connector_2.11 (version 2.0.5) to load data from Cassandra into Spark cluster. I am using read api to load the data as follows :

SparkUtil.initSpark()
         .read
         .format("org.apache.spark.sql.cassandra")
         .options(Map("table"-><table_name>, "keyspace"-><keyspace>))
         .load()

Its working fine, however, in one of the use case I want to read only a specific column from Cassandra. How to use read api to do the same?

Answer 1

SparkUtil.initSpark()
         .read
         .format("org.apache.spark.sql.cassandra")
         .options(Map("table"-><table_name>, "keyspace"-><keyspace>))
         .load()
         .select("column_name")

Use select.. you can also use case classes

Answer 2

Other way is to use following approach without using options api.

SparkUtil.initSpark()
         .sparkContext
         .cassandraTable(<keyspace>, <table_name>) 
         .select(<column_name>)

Answer 3

One line solution for fetching few columns from Cassandra table :

val rdd=sc.cassandraTable("keyspace","table_name")
.select("service_date","mobile").persist(StorageLevel.MEMORY_AND_DISK)