sstableloader of cassandra - java.lang.OutOfMemoryError

Question

I tried to use bulk load by sstableloader, and following "www.datastax.com/dev/blog/using-the-cassandra-bulk-loader-updated r" and "github.com/yukim/cassandra-bulkload-example/".

but got error when use sstableloader to load data, as below:

command: ./sstableloader -d localhost --debug -v ../bulkload_data/

Established connection to initial hosts
Opening sstables and calculating sections to stream
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335)
        at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:463)
        at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448)
        at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432)
        at org.apache.cassandra.io.sstable.SSTableReader.openMetadata(SSTableReader.java:225)
        at org.apache.cassandra.io.sstable.SSTableReader.openForBatch(SSTableReader.java:160)
        at org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:112)
        at java.io.File.list(File.java:1155)
        at org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:73)
        at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:155)
        at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:94)

I've tried to change the memory, accroding https://support.datastax.com/entries/68215305-sstableloader-throws-out-of-memory-error, but still error.

Data:

ll bulkload_data/

total 328
-rw-rw-r--. 1 cassandra cassandra    163 Apr 22 17:10 quote-historical_prices-ka-1-CompressionInfo.db
-rw-rw-r--. 1 cassandra cassandra 303506 Apr 22 17:10 quote-historical_prices-ka-1-Data.db
-rw-rw-r--. 1 cassandra cassandra     10 Apr 22 17:10 quote-historical_prices-ka-1-Digest.sha1
-rw-rw-r--. 1 cassandra cassandra     16 Apr 22 17:10 quote-historical_prices-ka-1-Filter.db
-rw-rw-r--. 1 cassandra cassandra    940 Apr 22 17:10 quote-historical_prices-ka-1-Index.db
-rw-rw-r--. 1 cassandra cassandra   4454 Apr 22 17:10 quote-historical_prices-ka-1-Statistics.db
-rw-rw-r--. 1 cassandra cassandra     99 Apr 22 17:10 quote-historical_prices-ka-1-TOC.txt

OS:

             total       used       free     shared    buffers     cached
Mem:         23944      13936      10007          0        413       4930
-/+ buffers/cache:       8592      15351
Swap:        49999        543      49456

cassandra version: apache-cassandra-2.0.13

node: 1

and I have another question, why sstableloader always asking "InvalidRequestException(why:No such keyspace: cassandra"...I think it should be use "quote"...

Fixed it after upgrade cassandra from apache-cassandra-2.0.13 to apache-cassandra-2.1.4. But still can't import the data to correct keyspace.. — chio

root root · Accepted Answer · 2016-02-05T22:34:17

You should have the keyspace and table already in Cassandra before you load using sstableLoader , also your sstableLocation must be like this.

e.g if you are trying to load to a keyspace "rambo" which has table "rocky", then your command should be

./sstableloader -d host /Path/bulkload_data/rambo/rocky/

so your actual sstable must be in directory named "rocky" which should be inside the "rambo"(same name as keyspace)

sstableloader of cassandra - java.lang.OutOfMemoryError

1 Answers