Transfer data from one column family db to another column family

0
votes

I'm new to Cassandra and Column family database world.I have a scenario where I need to move data from one Column family database such as Scylla Database to another Column family database Datastax Cassandra.Amount of data to be transferred will be in millions. And I wan this data transfer to happen on regular interval time lets say 2 mins.I was exploring sstableloader option. No luck yet. is ter any other better approach for my scenario ? Any suggetions will be highly appreciated.

2
cassandradatastax-enterprisecolumn-familyscyllabigdata
stackoverflow.com/questions/26724662/… – Balaji Reddy

2 Answers

2
votes

(Disclaimer: I'm a ScyllaDB employee)

There are 3 ways to accomplish this:

  1. Dual writes from the client side with client side time stamps, to both DBs
  2. Use the sstableloader tool to migrate the data from one DB to the other.
  3. Use nodetool refresh command to load sstables

You can read more about Migration from Cassandra to Scylla in the following doc, which also describes how to perform dual writes from the client side (option-1), with code examples + how to use the sstableloader tool (option-2)

http://docs.scylladb.com/procedures/cassandra_to_scylla_migration_process/

For the nodetool refresh usage you may look here: http://docs.scylladb.com/nodetool-commands/refresh/

0
votes

A common approach is to instrument the client to write to both databases in parallel, instead of synchronizing the two databases. This keeps the two databases in sync on every single write.