Unable to write entries in Cassandra when 2 out of 3 nodes down

1
votes

I'm inserting data in Cassandra via C++. I've got 2 datacentres and total 3 nodes distributed among the 2 datacentres(dc1 and dc2). As I'm doing some experiments, so I've stopped Cassandra on 2 of the nodes. Now when I try to write data into the one remaining node, I get errors like "no hosts available". Since in the C++ program, I've not mentioned any consistency for the write statement, then why is it not working. I had read that the default write consistency was local one, therefore I should be able to write data even when only one out of 3 nodes is up.

dc1 has 2 nodes and dc 2 has 1 node. replication strategy: network topology, dc1: 2, dc2: 1 Cassandra: 3.0.14 RHEL 6

Edit: The problem was sorted out after I changed 2 things and restarted my cluster: 1. I configured the cassandra-topology.properties. Initially it was not set with the right values. 2. I synced the time between the 3 nodes. I don't know which one(if not both) solved the problem.

2
c++cassandracassandra-3.0
What replication factor are you using? - Simon Fontana Oscarsson
It might take default consistency level from schema level. Please share more details with your CQL statements. - sayboras
Which two nodes did you shut down? - Simon Fontana Oscarsson
It doesn't matter which 2 nodes I shut down. I'm unable to enter data if only one node is up i.e. it requires 2 nodes to enter data. - Vishal Sharma
@VishalSharma but what RF are you using on each DC? - Simon Fontana Oscarsson

2 Answers

1
votes

RF does not matter for this problem. It depends on the write CL. I assume your write CL is LOCAL_ONE. Since you have only 1 up-node, it may be on dc1 or dc2. If your up-node is in dc1, since dc1 has two nodes, the hash-ring is shared by these two nodes. So, Cassandra should determine which node to write in dc1. Cassandra computes the hash of your data's partition key. If the up-node is the owner of that partition, the write succeeds, otherwise it fails.

If your up-node is in dc2, since dc2 has only 1 node, that node owns all the data. So the write should always succeed.

0
votes

If the keyspace you are writing to is set with RF > 1, than it means the data must be written in more than 1 replica (more than 1 node) and since only 1 node remains active, that does not satisfy the RF settings.

I assume this is the reason for the error you see.