Cassandra Consistency Level In Some Special Scenario

0
votes

Consider a cluster of 4 nodes.

The replication factor has been set to 3.

We gave a write query with consistency level set to ALL but only one of the replicas successfully wrote the data and the other two never responded. (crashed during write)

In this situation, the coordinator responds with TimeOutException so we think that our request failed to persist.

Then we gave a read query for the same row_key with consistency level set to 1, coordinator sends this request to the same node which was successfully written the previous request.

My question is what happens next? whether this read query returns with value? and if so does this behavior really make any sense because we threw UnavailableException in the first place?

1
cassandranosqlcqleventual-consistency
I was incorrect with my answer... I just checked the code - there is a check (in the StorageProxy.java) before writing the data to check if we're able to reach given CL at all. So in this case we shouldn't send write request to any node. – Alex Ott
You are right Alex but I am more interested in the situation in which the coordinator thinks that all nodes are fine but some of them suddenly crash during a write operation. – Hadi Moloodi
Ok, then it's a different situation, let me undelete my answer & modify it. – Alex Ott

1 Answers

1
votes

Answer was modified after clarification

Cassandra can answer early with UnavailableException if it has information (based on Gossip) that given CL won't be achieved as it doesn't have enough replicas.

Coordinator node will attempt to write to all required replicas, and if some of them crashed during the operation, then coordinator may write a hint for them, that will be replayed after nodes are back (during 3 hours by default). Coordinator node should return WriteTimeoutException or WriteFailureException.

Even if the nodes aren't back, and you're reading with ONE or LOCAL_ONE, then you can get data back. This is what called Eventual Consistency - data will be eventually propagated to all replicas - either via hints or via repairs.

You can find more information on how Cassandra writes & reads data with consistency levels in the DataStax Architecture Guide.