Gemfire Junit for multi module project tests are taking too long

Question

I have a project which is composed of several modules each module holds it's region and repository classes.

The problem with that is that each module has it's own Gemfire gfe:cache on it's own spring context.

So my problem is that when I run my mvn test every module starts it's own Gemfire and closes it after it's tests and this makes my test runt to take almost 10 minutes, every instance of Gemfire takes 40s to start.

So I would like to know what is the best way to avoid that?

I was thinking about having the parent module (holds all repositories and regions) holding and creating the regions and then using lookups on the children modules to use them. But I also need the individual modules to run by themsleves in case I would like to run only one of the modules tests.

Is there a way to use lookup and in case it fails create a cache with that same region lookedup? Or having the cache to be created once (first test) and get the regions added to it while the other contexts are started instead of being closed?

Thanks

Answer 1

In certain cases, you cannot avoid it, especially if you dirty the Spring context or your GemFire instance in such a way that it would lead to conflicts when running subsequent tests in your test suite.

I.e. it maybe more work to try to manage isolation/separation in each new test class and/or test case you write, having to keep track of what test touches what (e.g. Region), rather then perhaps just restarting the GemFire instance per test class (or in the worse case, per test case). E.g., think of a OQL Query pulling in unexpected results due to a previous test's actions, or something of that nature.

The Spring Data GemFire test suite is very similar in that it starts up GemFire instances and stops them, either per test case, or in most cases, per test class. The entire build with tests (900+ tests) runs on avg in ~15 minutes. GemFire's own test suite (unit + integration/distributed tests, etc) runs in about 8-12 hours depending on how effectively you can "parallelize" the tests, o.O

I am a firm believer in the 10 minute build, but GemFire is a complex beast and writing tests, especially distributed tests, effectively takes careful planning.

Mosts of the tests in Spring Data GemFire are peer cache applications (i.e. the test JVM is embeds the GemFire instance, for example... RegionDataPolicyShortcutsIntegrationTest and it's associated Spring context configuration file).

I have found that if you set of a few GemFire properties (such as setting log-level to "warning" and particularly mcast-port to 0) you can reduce the runtime substantially, for example. Setting mcast-port to 0 sets up a "loner" GemFire node and substantially improves startup time.

There are other Spring Data GemFire tests, which are "ClientCaches", that even spawn a separate JVM with a GemFire Server process to test the client/server interaction. You can imagine those taking even longer to start/stop, and in fact they do. For example, the ClientCacheFunctionExecutionWithPdxIntegrationTest and the associated Spring context configuration files... (server) and of course the (client). Note, the test is the GemFire ClientCache VM in this case.

Now, you might be able to get a way with use mocks in certain tests. Many of the Spring Data GemFire tests mock the interactions between SDG and GemFire using the mocks provided in the org.springframework.data.gemfire.test package.

The test classes in SDG that use mocks declare a special kind of Spring ApplicationContextInitializer like so in the GemfireTemplateTest, which uses the SDG GemfireTestApplicationContextInitializer. The real magic is in the GemfireTestBeanPostProcessor though. If you trace the code through, you will get the idea.

In time, I hope to formalize these mocks a bit and create a separate project with these mocks, the GemfireTestApplicatonContextInitializer and associated GemfireTestBeanPostProcessor for developers testing purposes involving Spring and GemFire (and also Apache Geode).

Hope this gives you a few ideas to ease the pain of the test setup and runtime.

Cheers!