Spring Data GemFire and loose coupling between GemFire cache with Spring Boot on startup

2
votes

We have a GemFire cluster with 2 Locators and 2 Cache nodes.

Our Spring Boot services will connect to the GemFire cluster as clients and will have client Regions. We are using Spring Data GemFire to bootstrap client Regions with GemFire XML config and properties.

When the GemFire cluster is down the Spring Boot service is not coming up as it couldn’t satisfy the GemFire Region dependencies (UnsatisfiedDependecyException) .

Is there a way to loosely couple Spring Boot startup and GemFire?

In essence, we want the Spring Boot service to start even when the GemFire cluster is down.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:gfe="http://www.springframework.org/schema/gemfire"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/gemfire http://www.springframework.org/schema/gemfire/spring-gemfire.xsd
                http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">

   <util:properties id="gemfireProperties" location="classpath:gemfire.properties"/>

     <bean id="autoSerializer" class="org.apache.geode.pdx.ReflectionBasedAutoSerializer">
        </bean>

    <gfe:client-cache pdx-serializer-ref="autoSerializer" pdx-read-serialized="true" pool-name="POOL" properties-ref="gemfireProperties"/>

<gfe:pool id="POOL" subscription-enabled="true" >
     <gfe:locator host="${gf.cache.locator1}" port="${gf.cache.locator1.port}"/>
     <gfe:locator host="${gf.cache.locator2}" port="${gf.cache.locator2.port}"/>
 </gfe:pool>

<gfe:client-region id="xyz" shortcut="CACHING_PROXY" pool-name="POOL">
    <gfe:regex-interest pattern=".*" result-policy="KEYS_VALUES"/>
    </gfe:client-region>


</beans>


@ImportResource({"classpath:gemfire-config.xml"})
1
spring-bootgemfirespring-data-gemfire

1 Answers

3
votes

What you are asking is possible to do, but not without some custom code.

And, it would be much easier to accomplish using Java-based, Spring Container Configuration along with SDG's API than using either Spring (Data GemFire) XML config or (Did I read this right?? You are (possibly) using...) GemFire XML config.

First, though, I wonder in what capacity are you using Pivotal GemFire that your Spring Boot applications (or services) do not strictly require GemFire to be running (server-side) to function properly and so that your Spring Boot apps/services will still startup and service your customers' needs?

Clearly, Pivotal GemFire is not being used as a System of Record (SOR) for your Spring Boot services in this case. However, it would make sense if you were simply using Pivotal GemFire for "caching", perhaps as a caching provider (or this) in Spring's Cache Abstraction? Is this what you are doing?

Anyway...

I think the best way to demonstrate this is by example with an Integration Test, ;-)

I wrote a simple Integration Test, ResilientClientServerIntegrationTests, where the test is functioning as an application (to put/get data to/from a Region, i.e. "Example") and demonstrates that it can "conditionally" switch between client/server and local-only mode.

The key to the test (or Spring-based application) to switch between client/server and local-only mode is by implementing a custom Spring Condition and then using the @Conditional Spring annotation on the application (client) configuration class, as shown here.

However, instead of completely disabling the GemFire client when the server cluster is not available, I simply switch the application (a.k.a. test) to run in client, local-only mode.

I specifically do this by configuring the client Regions to use the ClientRegionShortcut.LOCAL setting. I then use this setting in the configuration of my client-side GemFire objects, e.g. on the "Example" client Region, see here, then here.

Now, if I run this test, it will pass whether or not I have a GemFire cluster (of servers) running, because if there is no GemFire cluster available, then it will simply function in local-only mode.

If a GemFire cluster has been made available to the application, then it will also work as expected and use the cluster without changing any client application code or configuration, neat huh!

So, by way of example, suppose I start a cluster using Gfsh, like so...

$ echo $GEMFIRE
/Users/jblum/pivdev/apache-geode-1.6.0

$ gfsh
    _________________________     __
   / _____/ ______/ ______/ /____/ /
  / /  __/ /___  /_____  / _____  / 
 / /__/ / ____/  _____/ / /    / /  
/______/_/      /______/_/    /_/    1.6.0

Monitor and Manage Apache Geode
gfsh>


gfsh>start locator --name=LocatorOne --log-level=config
Starting a Geode Locator in /Users/jblum/pivdev/lab/LocatorOne...
.....
Locator in /Users/jblum/pivdev/lab/LocatorOne on 10.99.199.24[10334] as LocatorOne is currently online.
Process ID: 9737
Uptime: 3 seconds
Geode Version: 1.6.0
Java Version: 1.8.0_192
Log File: /Users/jblum/pivdev/lab/LocatorOne/LocatorOne.log
JVM Arguments: -Dgemfire.enable-cluster-configuration=true -Dgemfire.load-cluster-configuration-from-dir=false -Dgemfire.log-level=config -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/jblum/pivdev/apache-geode-1.6.0/lib/geode-core-1.6.0.jar:/Users/jblum/pivdev/apache-geode-1.6.0/lib/geode-dependencies.jar

Successfully connected to: JMX Manager [host=10.99.199.24, port=1099]

Cluster configuration service is up and running.


gfsh>start server --name=ServerOne --log-level=config
Starting a Geode Server in /Users/jblum/pivdev/lab/ServerOne...
....
Server in /Users/jblum/pivdev/lab/ServerOne on 10.99.199.24[40404] as ServerOne is currently online.
Process ID: 9780
Uptime: 3 seconds
Geode Version: 1.6.0
Java Version: 1.8.0_192
Log File: /Users/jblum/pivdev/lab/ServerOne/ServerOne.log
JVM Arguments: -Dgemfire.default.locators=10.99.199.24[10334] -Dgemfire.start-dev-rest-api=false -Dgemfire.use-cluster-configuration=true -Dgemfire.log-level=config -XX:OnOutOfMemoryError=kill -KILL %p -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/jblum/pivdev/apache-geode-1.6.0/lib/geode-core-1.6.0.jar:/Users/jblum/pivdev/apache-geode-1.6.0/lib/geode-dependencies.jar


gfsh>list members
   Name    | Id
---------- | ----------------------------------------------------------------
LocatorOne | 10.99.199.24(LocatorOne:9737:locator)<ec><v0>:1024 [Coordinator]
ServerOne  | 10.99.199.24(ServerOne:9780)<v1>:1025


gfsh>create region --name=Example --type=PARTITION
 Member   | Status
--------- | ----------------------------------------
ServerOne | Region "/Example" created on "ServerOne"


gfsh>list regions
List of regions
---------------
Example


gfsh>describe region --name=/Example
..........................................................
Name            : Example
Data Policy     : partition
Hosting Members : ServerOne

Non-Default Attributes Shared By Hosting Members  

 Type  |    Name     | Value
------ | ----------- | ---------
Region | size        | 0
       | data-policy | PARTITION

Now, I run the test again, it passes, and then I assess the state of the cluster:

gfsh>describe region --name=/Example
..........................................................
Name            : Example
Data Policy     : partition
Hosting Members : ServerOne

Non-Default Attributes Shared By Hosting Members  

 Type  |    Name     | Value
------ | ----------- | ---------
Region | size        | 1
       | data-policy | PARTITION



gfsh>get --region=Example --key=1 --key-class=java.lang.Integer
Result      : true
Key Class   : java.lang.Integer
Key         : 1
Value Class : java.lang.String
Value       : test

Cool! It worked! Our "Example" Region contains an entry put their by our test/application.

If I stop the cluster and re-run the test, of course, it will still pass because the code/configuration smartly switches back to local-only mode, seamlessly without doing anything.

If you are unclear/uncertain that the test is doing what I say it is doing, then simply comment out the @Conditional annotation that is responsible for A) determining whether the GemFire cluster is available and B) deciding how to handle the situation when the cluster is unavailable, which in this case we simply switch to local-only mode.

But, by commenting out that condition, you would see an Exception similar to the following:

org.apache.geode.cache.client.NoAvailableLocatorsException: Unable to connect to any locators in the list [LocatorAddress [socketInetAddress=localhost/127.0.0.1:10334, hostname=localhost, isIpString=false]]

    at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.findServer(AutoConnectionSourceImpl.java:158)
    at org.apache.geode.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:234)
    at org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:242)
    at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:148)
    at org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:127)
    at org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:782)
    at org.apache.geode.cache.client.internal.PutOp.execute(PutOp.java:91)
    at org.apache.geode.cache.client.internal.ServerRegionProxy.put(ServerRegionProxy.java:159)
    at org.apache.geode.internal.cache.LocalRegion.serverPut(LocalRegion.java:3010)
    at org.apache.geode.internal.cache.LocalRegion.cacheWriteBeforePut(LocalRegion.java:3121)
    at org.apache.geode.internal.cache.ProxyRegionMap.basicPut(ProxyRegionMap.java:239)
    at org.apache.geode.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5631)
    at org.apache.geode.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:152)
    at org.apache.geode.internal.cache.LocalRegion.basicPut(LocalRegion.java:5059)
    at org.apache.geode.internal.cache.LocalRegion.validatedPut(LocalRegion.java:1597)
    at org.apache.geode.internal.cache.LocalRegion.put(LocalRegion.java:1584)
    at org.apache.geode.internal.cache.AbstractRegion.put(AbstractRegion.java:413)
    at example.tests.spring.data.geode.clientserver.ResilientClientServerIntegrationTests.exampleRegionDataAccessOperationsAreSuccessful(ResilientClientServerIntegrationTests.java:86)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)

That is NoAvailableLocatorsException, because the "default" is PROXY (again, here) which expects that a cluster with the corresponding client Region (i.e. "Example") exists in the server cluster.

Of course, you can completely disable any GemFire client configuration in your [Spring [Boot]] application/services if you absolutely and strictly don't want any GemFire client objects functional when the cluster is not available. You would simply return false here. You just have to be careful that your application has not auto-wired any GemFire objects in this case, for example.

Also, you can accomplish a similar effect with Spring XML config as well, but using Java-based Spring configuration was much easier to demonstrate and I leave it as an exercise for you to figure out.

Additionally, the logic to test the availability of the cluster, while effective (and hardcoded, :P), is crude and I leave it to you to add more "robust" logic.

But, I trust this addresses your question adequately.

Hope this helps!

Cheers!