Unit testing spring batch jobs

Question

What would be the approach to test the below scenarios in Spring Batch jobs:

1) An independent tasklet running in a step in a spring batch job.

2) A job consisting of several steps with some steps having itemreader, itemprocessor, itemwriters while some steps having tasklets.

3) An itemreader where the input query to item reader is a very complex sql query having joins from many tables - This scenario will not allow emptying all tables used and populating them with dummy data for testing. In other words, there could be real world scenarios where in it would not be possible to empty the tables for testing. What should be the approach in that case ?

If I get a little help on this forum about the approach to be taken, I want to prepare an example suite containing examples on testing spring batch jobs with different job scenarios.

On a general note: make sure that you don't end up writing tests that are effectively tests for your database or for Spring Batch. This means that you can use the techniques below (i.e. use HSQLDB) to simulate a database and its tables. This can be done using an inmemory database and hence will be many times quicker than using a real database. — Fortyrunner

Matthew Farwell Matthew Farwell · Accepted Answer · 2012-10-30T10:05:53

It's very hard to 'unit' test a spring batch job, purely because there is so much interaction with external resources, such as databases or files etc. The approach I have has two levels:

The individual methods in the tasklets are unit tested as normal, I just call the methods directly from a JUnit test, with the usual mocks or stubs.

Secondly, I run the batch from a JUnit test, by calling the main method directly, with all of the necessary infrastructure in place. For my batch jobs, I need a database and some input files. My JUnit test copies all of the necessary files for that test into a temp directory (I use maven so usually target/it), along with a database (I have copies of my MySQL database saved in HSQLDB format). I then call Main.main() directly with the correct parameters, let the batch run and then check the results, check to see if the correct files have been generated, the database has been changed correctly etc.

This has a couple of advantages as a way of working.

I can run these tests from Eclipse, so shortening the debugging cycle, because I don't need to build the full package each time to test it.
They are easy to incoporate into the build, I just include them in the failsafe plugin in maven.

With a couple of warnings:

You'll need to run the main method within a SecurityManager. If your main calls System.exit(), you don't want the JVM to stop. For an example of a SecurityManager, see org.junit.tests.running.core.MainRunner. This is called like:

Integer exitValue = new MainRunner().runWithCheckForSystemExit(new Runnable() {
    public void run() {
    Main.main(new String[] {});
  }
});

Using the above, you can assert that a batch calls System.exit() with the correct values on failure, allowing testing of failure conditions as well.

Secondly, you can do a lot of setup from a JUnit test, but you can't do it all, so if for instance, you require an FTP server, I usually start it from maven rather than the JUnit, before the failsafe plugin runs. This can be a little bit fiddly sometimes.

Unit testing spring batch jobs

1 Answers