1
votes

Just looking for some information if others have solved this pattern. I want to use Spring Integration and Spring Batch together. Both of these are SpringBoot applications and ideally I'd like to keep them and their respective configuration separated, so they are both their own executable jar. I'm having problems executing them in their own process space and I believe I want, unless someone can convince me otherwise, each to run like they are their own Spring Boot app and initialize themselves with their own profiles and properties. What I'm having trouble with though is the invocation of the job in my SpringBatch project from my SpringIntegration project. At first I couldn't get the properties loaded from the batch project, so I realized I need to pass the spring.active.profiles as a Job Parameter and that seemed to solve that. But there are other things in the Spring Boot Batch application that aren't loading correctly like the schema-platform.sql file and the database isn't getting initialized, etc.

On this initial launch of the job I might want the response to go back to Spring Integration for some messaging on Job Status. There might be times when I want to run a job without Spring Integration kicking off the job, but still take advantage of sending statuses back to the Spring Integration project providing its listening on a channel or something.

I've reviewed quite a few Spring samples and have yet to find my exact scenario, most are with the two dependencies in the same project, so maybe I'm doing something that's not possible, but I'm sure I'm just missing a little something in the Spring configuration.

My questions/issues are:

  • I don't want the Spring Integration project to know anything about the SpringBatch configuration other than the job its kicking off. I have found a good way to do that reference to the Job Bean without getting my entire batch configuration loading.
  • Should I keep these two projects separated or would it be better to combine them since I have two-way communication between both.
  • How should the Job be launch from the integration project. We're using the spring-batch-integration project with JobLaunchRequest and JobLauncher. This seems to run it in the same process as the Spring Integration project and I'm missing a lot of my SpringBootBatch projects initialization
  • Should I be using a CommandLineRunner instead to force it to another process.
  • Is SpringApplication.run(BatchConfiguration.class) the answer?

Looking for some general project configuration setup to meet these requirements.

1
If you have 2 separate projects you need some communication mechanism like a queue between them. If you don't you are bound to have them as a single artifact. You can put JobLaunchRequest instances on a queue which is read on the Spring Batch side (with Spring Integration probably so you still need a part of that on the batch side) to launch the job.M. Deinum
So it sounds like to really decouple the environments, I have one Spring Boot Spring Integration project (aka ProjectA), and one Spring Boot Spring Batch environment (with an embedded Spring Integration project) to launch the jobs (ProjectB).Dale Highfill
ProjectA in this case is pretty lightweight, it just knows about SpringIntegration configuration, file polling and the spring.active.profiles that would need to be sent to ProjectB. I assume that's a Spring Message that I send that would contain these batch specific parameters.Dale Highfill
From my testing even though I was passing the active profiles and other job parameters (e.g. inputFile=myfile.xt) I was not able to get the Batch project to recognize those on startup.Dale Highfill
And why would it? It is launching jobs it isn't starting a container, so passing active profiles isn't going to do much.M. Deinum

1 Answers

0
votes

Spring Cloud Data Flow in combination with Spring Cloud Task does exactly what you're asking. It launches Spring Cloud Task applications (which can contain batch jobs) as new processes on the platform you choose. I'd encourage you to check out that project here: http://cloud.spring.io/spring-cloud-dataflow/