Not able to use a java class for mapping in spark-cassandra-connector

0
votes

I'm trying to use a java class in mapping a cassandra table to spark rdd, for which I'm facing an issue that cassandra is not able to create a query to fetch the data.

According to this link https://github.com/datastax/spark-cassandra-connector/blob/master/doc/6_advanced_mapper.md, a class must bu serializable and have 'getters' and 'setters' for at least the primary key columns of the table you are trying to map.

So what I have done is, I have created a java class, created public getters and setters for each field, mapped the class to scala companion object and used it for mapping a cassandra table while retrieving the data into an rdd.

I have added code in the image links.

  1. Java class
  2. Mapper
  3. Retrieving data
  4. Exception
  5. Scala classes which give no error

All of the above works seamlessly when I use scala classes

1
apache-sparkcassandracassandra-3.0spark-cassandra-connector
why aren't you using Scala's case classes if you use Scala API? - Alex Ott
I actually need to use those java classes for an 'event simulator' and also for 'api' request and response and want to maintain a same entity base for all these components, hence I'm trying to figure out a way to get it done in this manner. - Omkar Rahane
What is the schema for a table? - Alex Ott
This is the create statement if the table: CREATE TABLE eventsfdd.events (deviceid uuid,occurancetime timestamp,eventid uuid,buildingid uuid,cityid uuid,description text,devicetype text,emittime timestamp,emittimeutc timestamp,eventtype text,faultid uuid,floorid uuid,inserttime timestamp,inserttimeutc timestamp,isnewevent boolean,kafkareceivedtime timestamp,kafkareceivedtimeutc timestamp,occurancetimeutc timestamp,regionid uuid,severity text,status text,streamreceivedtime timestamp,streamreceivedtimeutc timestamp,PRIMARY KEY (deviceid, occurancetime, eventid) - Omkar Rahane
What is the version of connector? - Alex Ott

1 Answers

0
votes

it won’t work for you by design - the class that you are providing doesn’t have any of fields that you have in your table. The mapper class analyzes your POJO class and extracts all getters and setters and use them to find names of the fields in the database. mapper is dropping the get/set/is prefixes from method names, and lowercase the rest of the method’s name and use it as a field name.

You need to define your POJO class to have fields (and setters/getters) format least the fields of primary key of the table.