Modularize Job definition

1
votes

Here is what I am trying to achieve:

  1. A batch job server which provides a unify entry point for jobs of my applications and a centralized place for jobs to run, which is built on top of Spring Batch
  2. Different applications can create their jobs bundle and "deploy" under the batch job server. The "bundle" will contains the job definitions, dependency JARs etc.
  3. The application job bundle should be self-contained: 2 job bundles can have different version of same class (i.e. better to have separate classloaders for different bundles), so that developer will seldom need to worry about jobs of other applications.
  4. Hot deployment is NOT required. I can accept to stop my batch job server application, deploy the new bundle, do some config, and start the server again.

In brief, it is like a plugin-in system for Spring Batch jobs.

Until now, the way I am using Spring Batch is to build an application, with job definition being part of the application (either be part of the application source, or as dependencies). However when I am trying to have one batch server serving multiple applications, I fall into problems like, when application 1 needs to have minor change in its job, I will need to release the whole server (which includes jobs from other applications). I would want to see if:

  1. Is there any existing solution by Spring Batch etc?
  2. If there is no existing solution and I need to invent the wheel, what technologies should I take a look on? ServiceLoader from Java? OSGi?
1
javaspring-batchspring-xd

1 Answers

2
votes

What it sounds like you're looking for is Spring XD (http://projects.spring.io/spring-xd/). To go through your items point by point:

  1. A batch job server which provides a unify entry point for jobs of my applications and a centralized place for jobs to run, which is built on top of Spring Batch - Spring XD is a distributed runtime that is built upon Spring Batch and Spring Integration. It provides a central place for managing job deployments, where they are deployed to, launching them, browsing the job repository, etc. You can interact it via an interactive shell, a web UI, or the REST API that both the previous options use.
  2. Different applications can create their jobs bundle and "deploy" under the batch job server. The "bundle" will contains the job definitions, dependency JARs etc. - Spring XD uses Spring Boot as the packaging mechanism. Using a special plugin (maven and gradle are both supported), you package your job into a fat jar just like you would to execute it as an executable Spring Boot jar containing a job definition and all it's required dependencies. While currently the structure is one job per artifact (so one job per Spring Boot jar), the Spring XD team is working on being able to move multiple modules (jobs in your case) around in larger groupings.
  3. The application job bundle should be self-contained: 2 job bundles can have different version of same class (i.e. better to have separate classloaders for different bundles), so that developer will seldom need to worry about jobs of other applications. - All of this is supported in Spring XD. Each module gets it's own classloader and can specify it's own independent dependencies.
  4. Hot deployment is NOT required. I can accept to stop my batch job server application, deploy the new bundle, do some config, and start the server again. Hot deployment is supported in Spring XD.

I highly encourage you to take a look at Spring XD for what you're looking for. It sounds exactly like what you want.