Cassandra behavior on contact point based on data center

Question

Cassandra setup in 3 data-center (dc1, dc2 & dc3) forming a cluster

Running a Java Application on dc1. dc1 application has Cassandra connectors pointed to dc1 (ips of cassandra in dc1 alone given to the application)

turning off the dc1 cassandra nodes application throws exception in application like

All host(s) tried for query failed (no host was tried)

More Info:

cassandra-driver-core-3.0.8.jar
netty-3.10.5.Final.jar
netty-buffer-4.0.37.Final.jar
netty-codec-4.0.37.Final.jar
netty-common-4.0.37.Final.jar
netty-handler-4.0.37.Final.jar
netty-transport-4.0.37.Final.jar

Keyspace : Network topology
Replication : dc1:2, dc2:2, dc3:2
Cassandra Version : 3.11.4

do you use default load balancing policy? do you set local data center in driver explicitly? do you use local consistency level? — Alex Ott
@AlexOtt No LoadBalancingPolicy mentioned in Application code. — Vigneshwaran
Is there any version mismatch in the driver (Cassandra: 3.11.4 & cassandra-driver-core-3.0.8.jar) — Vigneshwaran

Jim Wartnick Jim Wartnick · Accepted Answer · 2019-04-12T17:58:54

Here are some things I have found out with connections and Cassandra (and BTW, I believe Cassandra has one of the best HA configurations of any database I've worked with over the past 25 years).

1) Ensure you have all of the components specified in your connection connection. Here is an example of some of the connection components, but there are others as well (maybe you've already done this):

 cluster = Cluster.builder()
    .addContactPoints(nodes.split(","))
    .withCredentials(username, password)
    .withPoolingOptions(poolingOptions)
    .withLoadBalancingPolicy(
            new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder()
                    .withLocalDc("MYLOCALDC")
                    .withUsedHostsPerRemoteDc(1)
                    .allowRemoteDCsForLocalConsistencyLevel()
                    .build()
               )
    ).build();

2) Unless the entire DC you're "working in" is down, you could receive errors. Cassandra doesn't fail over to alternate DCs unless every node is down in the DC. If less than all nodes are down and your client can't satisfy the client CL settings, you will receive errors. I was actually hoping, when I did testing a while back, that if you couldn't achieve client CL in the LOCAL DC (even if some nodes in the current DC were up) and alternate DCs could, that it would automatically fail over, but this is not the case (since I last tested).

Maybe that helps?

-Jim

Cassandra behavior on contact point based on data center

1 Answers