Cassandra query results vary

0
votes

I have 4-node 2-dc cassandra cluster, and there I have faced an unexplainable behaviour.

dc1 - node1, node2, node3.

dc2 - node4.

In my database I have columnFamily1 with replication factor dc1:2 dc2:1. The column family contains 28 columns, and 11 columns are in primary key. I query node4 from dc2, that contains 100% data. So, the problem is:

'Select * from columnFamily' returns 5103 records.

'Select (any column from primaryKey) from columnFamily' returns 733 records.

If I export columnFamily to CSV and then import to truncated table, the problem disappears. Can anyone explain, how could this happen? Is there any solution without truncating columnFamily?

1
cassandracentos
what consistency level are you using for inserts and queries? - Chris Lohfink
i use consistency level one, and query the node that owns 100% data. - I.Kapralov
unless you run a repair theres no way to know which nodes have the data after inserting with CL.ONE. The owns from status just describes how much of the ring its responsible, not if it actually got the data. If you do reads and write with CL.QUORUM (non-local) it will work like you expect. - Chris Lohfink

1 Answers

0
votes

So it looks like from your above statements, putting aside your comment (the portion about the "node that has 100% of the data"), you must have some consistency issues. As you're using consistency one, you will query one node for the details. Which node could be any one of them. The fact that you do "*" or "any column from primary key" shouldn't matter, but each time you run the query, you could get a different node for the results. If you're using CQLSH, you could try to log onto each node, one at a time, set your consistency to LOCAL_ONE and run a "select " or "select count()" and see what results you get. I'm guessing it would be different. As you have truncated the table and re-loaded it, that could clean up the consistency problems. Another approach you could have done is run repair on the column family and then re-try your experiment. What did you do to export the data? Did you use the CQLSH COPY command? If so, that will query all nodes for the data. Did you count the number of rows in the CSV? If so, did it match any of the counts from the queries themselves?