IBM DataStage load data to Apache cassandra throws TransportException

Question

I am using DataStage 11.7 cassandra connector to connect 2 nodes configured cassandra VMs (192.168.3.240 & 192.168.3.241).

My apache cassandra version is 3.11.3, and I am using datastax drivers: dse-java-driver-core-1.8.1.jar & dse-java-driver-extras-1.8.1.jar.

Reading data from cassandra connector in DataStage is fine, but when I try to insert data to cassandra table, it throws the following exception:

Cassandra_Connector_0,1: com.datastax.driver.core.exceptions.TransportException: [/192.168.3.241:9042] Connection has been closed
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 29)
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 15)
at com.datastax.driver.core.DriverThrowables.propagateCause (DriverThrowables.java: 26)
at com.datastax.driver.core.AbstractSession.prepare (AbstractSession.java: 77)
at com.ibm.is.cc.cassandra.engine.CassandraSession.prepareInsertStatement (CassandraSession.java: 161)
at com.ibm.is.cc.cassandra.engine.InsertMutator.<init> (InsertMutator.java: 25)
at com.ibm.is.cc.cassandra.config.MutationType$1.newMutator (MutationType.java: 21)
at com.ibm.is.cc.cassandra.engine.CassandraSession.getMutator (CassandraSession.java: 156)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepareColumnExtractor (CassandraConsumer.java: 92)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepare (CassandraConsumer.java: 62)

Any idea why it throws such exception?

Update 1: Now I am using the OSS drivers, the "insert data" job is working fine now. However, this time the "read data" job throws the NoSuchMethodError if I use the OSS drivers:

Cassandra_Connector_0,1: com.ascential.e2.common.CC_Exception: "Connector encountered an exception: java.lang.NoSuchMethodError: com/datastax/driver/core/SimpleStatement.setRoutingToken(Lcom/datastax/driver/core/Token;)Lcom/datastax/driver/core/RegularStatement; (loaded from file:/home/dsadm/cassandra-client-write/cassandra-driver-core-3.7.1.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75) called from class com.ibm.is.cc.cassandra.engine.CassandraSession (loaded from file:/opt/IBM/InformationServer/Server/DSEngine/../DSComponents/bin/cassandra/CassandraConnectorCore.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75).
    at com.ibm.is.cc.cassandra.engine.CassandraSession.executeSelect(CassandraSession.java:126)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.runProducers(CassandraAdapter.java:106)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.run(CassandraAdapter.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod(Delegator.java:40)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100(Delegator.java:16)
    at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke(Delegator.java:56)
    at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run(CassandraAdapterWrapper.java:169)
"
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.createException (CC_CassandraErrorCode.java: 124)
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.newWrapperFailureException (CC_CassandraErrorCode.java: 42)
at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod (Delegator.java: 44)
at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100 (Delegator.java: 16)
at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke (Delegator.java: 56)
at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run (CassandraAdapterWrapper.java: 169)

Do you see any errors in the system.log file on both nodes at time of query? — Alex Ott

Alex Ott Alex Ott · Accepted Answer · 2019-06-27T09:35:34

There is a probability that the problem is in drivers that you're using. You're using drivers for DSE, while for open source Cassandra you need to use OSS driver, with following Maven coordinates:

<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-core</artifactId>
  <version>3.7.1</version>
</dependency>
<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-extras</artifactId>
  <version>3.7.1</version>
</dependency>

IBM DataStage load data to Apache cassandra throws TransportException

1 Answers