How to Copy/Move data between columnfamilies with same structure in cassandra

2
votes

I'm looking for a method to copy/move data in a column family to another with same structure, in same keyspace.

I had already tried the COPY command to import as csv & export the same to destination column family.

Since the data is large, I'm getting timeouts while using COPY.

One method is to write an application to copy data using a client api and add to the other column family.

Is there any tool to copy/move data between different column family? Or any other way?

2
cassandrabigdatacqldata-migration

2 Answers

3
votes

I would recommend using Spark for this kind of bulk migration. It's also a useful tool for general maintenance of C*.

https://github.com/datastax/spark-cassandra-connector

With spark the command

sc.cassandraTable("ks1","table").saveToCassandra("ks2","table")

you would move your tables.

If you aren't interested in Spark I think a custom java program or Brian Hess's Bulkloader tool would be useful

https://github.com/brianmhess/cassandra-loader

0
votes

If you want to get rid of the timeouts of cqlsh. You could actually start cqlsh with the option --request-timeout=3600, which changes the timeout from the default 10 seconds to an hour.