Insert order and pagination with Cassandra

Question

Let me start from saying that I'm not very experienced with Cassandra and there is a chance I'm trying to use it for an incorrect task.

I'm having a piece of code which stores URLs I want to visit

List<URL> urls = new ArrayList();

The problem is that I'm storing lots of URLs and my application runs of of memory. In fact in don't want to keep this list in memory because it will grow to gigabytes. Cassandra appears to be the best solution because it can store data sets which are greater than available memory.

I created a simple table

CREATE TABLE links (
    url text,
    PRIMARY KEY(url));

and I was trying to get the next URL like this

SELECT * FROM url WHERE token(url) < ? LIMIT 1

Where ? is the recently visited URL.

The problem is that if I insert new URLs before running the above query those URLs might be inserted "below" the recently visited site. My application will miss them and will never go to those pages.

My question is how paginate though a growing table (while it's growing) and not miss anything?

Mikhail Stepura Mikhail Stepura · Accepted Answer · 2014-08-20T16:59:53

url in your case is a partition key, and defines a node on which the row (partition) will be stored. It doesn't define an order of rows (partitions).

http://cassandra.apache.org/doc/cql3/CQL.html#createTablepartitionClustering

Insert order and pagination with Cassandra

2 Answers