Ignite Thin Client in Kubernetes

2
votes

I'm trying to set up a distributed cache using Ignite and my java app through a thin client in a Kubernetes environment.

In my Kubernetes cluster, I have 2 pods with the java app and 2 pods of ignite. For the java pods to communicate with ignite pods, I have configured a thin client to connect with the ignite kubernetes service. With this configuration, I was expecting that the load balancing was on the kubernetes side. Here's what I have done in java code:

ClientConfiguration cfg = new ClientConfiguration()
                    .setAddresses("ignite-service.default.svc.cluster.local:10800")
                    .setUserName("user")
                    .setUserPassword("password");
IgniteClient igniteClient = Ignition.startClient(cfg);

While storing and getting objects from ignite, I deleted one of the ignite pods and, after a while, I was getting errors saying that "Ignite cluster is unavailable":

org.apache.ignite.client.ClientConnectionException: Ignite cluster is unavailable

With this behavior, I assume that the method setAddresses in ClientConfiguration class stores one of the IPs of the pods and channels all communication to that pod.

Is this what's happening in this method?

Ignite version 2.7

Kubernetes version 1.12.3

2
ignite

2 Answers

0
votes

You need to pass several IP addresses to enable the failover (aka. automatic reconnect) on the thin client end. Find more details here.

0
votes

Although you might have resolved the issue since the question was posted a long time back, but still putting an answer here for others.

With the Apache Ignite version(2.7+), you can modify your deployment to use Kubernetes IP Finder. With this Kubernetes will take care of discovering and connecting all server and client nodes.

TcpDiscoveryKubernetesIpFinder module will help you achieve this.

This is the discovery SPI that needs to be added to your configuration (Replace with appropriate Namespace and Service Name)

    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                    <constructor-arg>
                        <bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
                            <property name="namespace" value="default" />
                            <property name="serviceName" value="ignite" />
                        </bean>
                    </constructor-arg>
                </bean>
            </property>
        </bean>
    </property>

Official documentation can be found here - https://ignite.apache.org/docs/latest/installation/kubernetes/amazon-eks-deployment