Spring Batch: dynamic composite reader/processor/writer

0
votes

I've seen this (2010) and this (SO, 2012), but still have not got the answer I need...

Is there an option in Spring Batch to have a dynamic composite reader/processor/writer?

The idea is to have the ability to replace processor at runtime, and in case of multiple processors (AKA composite-processor), to have the option to add/remove/replace/change order of processors. As mentioned, same for reader/writer.

I thought of something like reading the processors list from DB (using cache?) and there the items (beans' names) can be changed. Does this make sense?

EDIT - why do I need this?

There are cases that I use processors as "filters", and it may occur that the business (the client) may change the requirements (yes, it is very annoying) and ask to switch among filters (change the priority).
Other use case is having multiple readers to get the data from different data warehouse, and again - the client changes the warehouse from time to time (integration phase), and I do not want my app to be restarted each and every time.
There are many other use cases, of course. plus this.

Thanks

1
spring-batch
What's the intend of doing that? Usually batch jobs are static. Maybe Spring Batch is not the right tool. If you tell me a bit more of your requirements I can advise. – Simon Martinelli
there are cases that I use processors as "filters", and it may occur that the business (the client) may change the requirements (yes, it is very annoying) and ask to switch among filters (change the priority). Other use case is having multiple readers to get the data from different data warehouse, and again - the client changes the warehouse from time to time (integration phase). so I do not want my app to be restarted each and every time. – OhadR
but then you don't need to exchange the processor. But your processor can choose what kind of code to execute. and also in the reader you can change the data source – Simon Martinelli
@simas_ch so u suggest a single processor that will dynamically decide what code to run. what is the difference then? we still end up with the same challenge... – OhadR
why? you can't create the batch job dynamically but if you have the processor that can handle everything dynamically. you could even read the code from the database. – Simon Martinelli

1 Answers

0
votes

I've started working on this project:

https://github.com/OhadR/spring-batch-dynamic-composite

that implements the requirements in the question above. If someone wanna contribute - feel free!