import huge batch data file into apache cassandra automatically

0
votes

I have 1198 CSV file, each more than one gigabyte. I want to import all of them into Apache Cassandra table. Is there a way to do it automatically?

1
cassandrainsert
are all of the CSV are same in term of schema and data type .? - Dileep
Do you want all the csv imported into a single table ? - Ashraful Islam
both of the questions are yes. - Sarah K

1 Answers

1
votes

If you want to import all your csv into a single table, then move all your csv to a directory. i.e data Now you can import the full directory into a table let's say data_table

COPY data_table FROM 'data/*';

By the way, you are importing huge amount of data, you should

  • Generating SSTable using CQLSSTableWriter
  • Loading existing SSTables using sstableloader