Cassandra TokenRangeOfflineException in a 5n cluster

Question

I'm new to Cassandra and facing the following TokenRangeOffline Exceptions, I've a 5n cluster with replication factor set to 5 and consistency level as Quorum.

I did a describe keyspace on my cluster :

CREATE KEYSPACE "StorageOS" WITH replication = {
    'class': 'NetworkTopologyStrategy',
     'DC'  : '5'
};

Exceptions :

2018-06-22 18:44:26,161 [DriverUpdateThead_139] ERROR  CountingConnectionPoolMonitor.java (line 94) com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=xx.xx.xx.xx(xx.xx.xx.xx):9160, latency=8(5009), attempts=2]UnavailableException()
com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=xx.xx.xx.xx(xx.xx.xx.xx):9160, latency=8(5009), attempts=2]UnavailableException()
    at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:165)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65)
    at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1$2.execute(ThriftColumnFamilyQueryImpl.java:190)
    at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1$2.execute(ThriftColumnFamilyQueryImpl.java:182)
    at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151)
    at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:119)
    at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:338)
    at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1.execute(ThriftColumnFamilyQueryImpl.java:180)

Caused by: UnavailableException()
    at org.apache.cassandra.thrift.Cassandra$get_slice_result$get_slice_resultStandardScheme.read(Cassandra.java:11815)
    at org.apache.cassandra.thrift.Cassandra$get_slice_result$get_slice_resultStandardScheme.read(Cassandra.java:11773)
    at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:11699)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
    at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:671)
    at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:655)
    at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1$2.internalExecute(ThriftColumnFamilyQueryImpl.java:195)
    at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1$2.internalExecute(ThriftColumnFamilyQueryImpl.java:182)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60)
    ... 20 more

Some more info :

Connected to StorageOS at localhost:9160.

[cqlsh 4.1.1 | Cassandra 2.1.11 | CQL spec 3.1.1 | Thrift protocol 19.39.0]

I know there are some similar questions asked which relate to this exception but in my specific case I'm not able to understand how to proceed. Can someone please help???

Aaron Aaron · Accepted Answer · 2018-06-23T13:59:19

One thing to look at, is which version of Cassandra you are running. Astyanax defaults to a "target" Cassandra version of 1.1. This is because the original code base leveraged the Thrift API, which uses port 9160 (as I can see you are).

But the newer versions of Cassandra (2.2+) disable the Thrift protocol by default. So you have two options:

1. Tell Astyanax about which version you are using (as indicated in the documentation):

AstyanaxContext<Keyspace> context = new AstyanaxContext.Builder()
    .withAstyanaxConfiguration(new AstyanaxConfigurationImpl()      
        .setCqlVersion("3.4.4")
        .setTargetCassandraVersion("3.11.2"))

Note that with this option, you'll want to set 9042 as the port.

Or

2. Enable Cassandra to listen on Thrift, by altering that setting in the cassandra.yaml (on all nodes).

# Whether to start the thrift rpc server.
start_rpc: true

See if that helps.

Cassandra TokenRangeOfflineException in a 5n cluster

1 Answers