Cassandra Container DB NoNodeAvailableException

Question

I use Cassandra docker image, spring boot services running on docker container, and use Datastax Java driver.

I can connect connect Cassandra and also see logs, with the below docker compose file,

version: '3.5'

networks:
  my-network:

 cassandra-db:
    container_name: cassandra-db
    image: cassandra:4.0-beta1
    ports:
      - "9042:9042"
    restart: on-failure
    environment:
      - CASSANDRA_CLUSTER_NAME='cassandra-cluster'
      - CASSANDRA_NUM_TOKENS=256
      - CASSANDRA_RPC_ADDRESS=0.0.0.0
      - CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
    volumes:
      - ./out/cassandra_data:/var/lib/cassandra
    networks:
      - my-network


 client-service:
    container_name: client-service
    image: data/client-service
    environment:
      - SPRING_PROFILES_ACTIVE=dev
    ports:
      - 8087:8087
    depends_on:
      - cassandra-db
    links:
      - cassandra-db
    networks:
      - my-network

My keyspace query is,

CREATE KEYSPACE IF NOT EXISTS SensorKeySpace WITH replication = {'class':'NetworkTopologyStrategy','replication_factor':3}

and building Cqlsession using,

.addContactEndPoint(new DefaultEndPoint(
                        InetSocketAddress.createUnresolved("cassandra-db",9042)))
                .withLocalDatacenter("DC1")
                .withKeyspace(CassandraConstant.KEY_SPACE_NAME.getValue())
                .build();

The Cassandra is up and running, and verified it,

CONTAINER ID        IMAGE         COMMAND                  CREATED             STATUS              PORTS                                  NAMES
57d1d29dbf00  client-service      "java -Djava.securit…"   4 minutes ago       Up 4 minutes   0.0.0.0:8087->8087/tcp                      client-service
00548a4058e6  cassandra:4.0-beta1  "docker-entrypoint.s…"   4 minutes ago  Up 4 minutes   7000-7001/tcp, 7199/tcp, 9160/tcp, 0.0.0.0:9042->9042/tcp   cassandra-db

The problem was in the beginning,

I used SimpleStrategy to create Keyspace and got the below error,

Caused by: [com.datastax.oss.driver.api.core.CqlSession]: Factory method 'session' threw exception; nested exception is java.lang.IllegalStateException: Since you provided explicit contact points, the local DC must be explicitly set (see basic.load-balancing-policy.local-datacenter in the config, or set it programmatically with SessionBuilder.withLocalDatacenter). Current contact points are: Node(endPoint=cassandra-db:9042, hostId=729bd04f-b345-4df3-a148-80545d30e032, hashCode=5563e4c4)=datacenter1. Current DCs in this cluster are: datacenter1

Then configured .withLocalDatacenter("DC1") and - CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch in docker compose.

After configuring, NO ERROR during startup the container but when i hit the query, i got the below error,

The Select Query is [SELECT NAME,VALUE FROM READING ]
client-service       | 2020-09-08 11:12:38.480 ERROR 1 --- [nio-8087-exec-3] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.util.concurrent.ExecutionException: org.springframework.data.cassandra.CassandraConnectionFailureException: Query; CQL [com.datastax.oss.driver.internal.core.cql.DefaultSimpleStatement@1265401a]; No node was available to execute the query; nested exception is com.datastax.oss.driver.api.core.NoNodeAvailableException: No node was available to execute the query] with root cause
client-service       | com.datastax.oss.driver.api.core.NoNodeAvailableException: No node was available to execute the query

I read the API doc, https://docs.datastax.com/en/drivers/java/4.0/com/datastax/oss/driver/api/core/NoNodeAvailableException.html which tell me

This can happen if all nodes are down, or if all the contact points provided at startup were invalid.

My Cassandra DB is up(verified), and connection is success to it. Anything is missing here?

Aaron Aaron · Accepted Answer · 2020-09-08T15:23:32

So the problem here, is the keyspace definition.

CREATE KEYSPACE IF NOT EXISTS SensorKeySpace WITH replication = 
    {'class':'NetworkTopologyStrategy','replication_factor':3}

The NetworkTopologyStrategy is data center aware. This means that it requires the replication to be set per data center. From what I can see, it looks like your code is looking for the "DC1" data center.

.withLocalDatacenter("DC1")

But then, your logs are telling you:

Current DCs in this cluster are: datacenter1

This means that your keyspace definition should look like this:

CREATE KEYSPACE IF NOT EXISTS SensorKeySpace WITH replication = 
    {'class':'NetworkTopologyStrategy','datacenter1':3}

And your code should be:

.withLocalDatacenter("datacenter1")

Note: Data center names are case-sensitive. Be sure to verify the data center name you are using in the cassandra-rackdc.properties file.

Cassandra Container DB NoNodeAvailableException

1 Answers