Failed to find affinity server node with data storage configuration for starting cache

Question

I'm trying to store Spark in-memory table into Ignite. When I try to do that I gets an error message as

Failed to find affinity server node with data storage configuration for starting cache [cacheName=SQL_PUBLIC_JSON_TBL, aliveSrvNodes=[]].

I'm running it in a HDP cluster setup in a Ec2 machine but when I do the same in the cluster machine here it works perfectly but not in the EC2 machine.

Thanks in advance.

UPDATE:

I'm using Spark shell. Here's the code.

val df = sqlContext.read.json("~/responses")
val s = df.select("response.id","response.name")
s.write.format(IgniteDataFrameSettings.FORMAT_IGNITE).option(IgniteDataFrameSettings.OPTION_CONFIG_FILE, "~/apache-ignite-fabric-2.6.0-bin/examples/config/spark/example-shared-rdd.xml").option(IgniteDataFrameSettings.OPTION_CREATE_TABLE_PRIMARY_KEY_FIELDS,"id").option(IgniteDataFrameSettings.OPTION_TABLE, "json_table").save()

Here is the xml config file that I use for my single Ignite server:

 <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="
            http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd">
        <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
            <property name="cacheConfiguration">
                <!-- SharedRDD cache example configuration (Atomic mode). -->
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <!-- Set a cache name. -->
                    <property name="name" value="sharedRDD"/>
                    <!-- Set a cache mode. -->
                    <property name="cacheMode" value="PARTITIONED"/>
                    <!-- Index Integer pairs used in the example. -->
                    <property name="indexedTypes">
                        <list>
                            <value>java.lang.Integer</value>
                            <value>java.lang.Integer</value>
                        </list>
                    </property>
                    <!-- Set atomicity mode. -->
                    <property name="atomicityMode" value="ATOMIC"/>
                    <!-- Configure a number of backups. -->
                    <property name="backups" value="1"/>
                </bean>
            </property>
            <property name="discoverySpi">
                <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                    <property name="ipFinder">
    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                            <property name="addresses">
                                <list>
                                    <!-- In distributed environment, replace with actual host IP address. -->
                                    <value>127.0.0.1:47500..47509</value>
                                </list>
                            </property>
                        </bean>
                    </property>
                </bean>
            </property>
        </bean>
    </beans>

Here's the full log.

18/10/30 12:32:54 WARN GridDiagnostic: Initial heap size is 252MB (should be no less than 512MB, use -Xms512m -Xmx512m). 18/10/30 12:32:54 WARN TcpCommunicationSpi: Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides. 18/10/30 12:32:55 WARN NoopCheckpointSpi: Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation) 18/10/30 12:32:55 WARN GridCollisionManager: Collision resolution is disabled (all jobs will be activated upon arrival). 18/10/30 12:32:57 WARN TcpDiscoverySpi: Failed to read message due to ClassNotFoundException (make sure same versions of all classes are available on all nodes) [rmtNodeId=3085dfa9-58ba-4ac0-a7f8-f78e2901a699, err=o.a.i.i.processors.hadoop.HadoopAttributes] 18/10/30 12:32:57 WARN IgniteAuthenticationProcessor: Cannot find the server coordinator node. Possible a client is started with forceServerMode=true. Security warning: user authentication will be disabled on the client. 18/10/30 12:32:58 ERROR ClusterCachesInfo: Failed to find affinity server node with data storage configuration for starting cache [cacheName=SQL_PUBLIC_JSON_TBL6, aliveSrvNodes=[]] 18/10/30 12:32:58 WARN CacheAffinitySharedManager: No server nodes found for cache client: SQL_PUBLIC_JSON_TBL

Please provide used Ignite configuration and DDL statement. Also it would be nice if you provided log, preceding this error message. — Denis

Denis Denis · Accepted Answer · 2018-10-30T13:05:53

The log contains the following record:

12:32:57 WARN TcpDiscoverySpi: Failed to read message due to ClassNotFoundException (make sure same versions of all classes are available on all nodes) [rmtNodeId=3085dfa9-58ba-4ac0-a7f8-f78e2901a699, err=o.a.i.i.processors.hadoop.HadoopAttributes]

It says, that a discovery message cannot be deserialized because HadoopAttributes class is not on the classpath. It may lead to connectivity problems and affect ability of nodes to see each other.

Make sure, that all nodes have ignite-hadoop module on their classpath, or get rid of this dependency.

Failed to find affinity server node with data storage configuration for starting cache

1 Answers