Spring Batch - construct 2 reader and 2 processor with single writer

Question

My case is I got a batch job will read data from 2 differents table and process differently.

The first reader will do simple SQL retrieving and simple conversion, the second reader will do SQL retrieving and process update and insert logic behind. Both readers will return a string line and write into a file.

In Spring Batch, possible to have 2 readers and 2 processor in 1 step then pass to 1 writer?

Answer 1

I'd go for the second approach, suggested by Faiz Pinkman. It's simply closer to the way spring-batch works.

first step

• reader for your simple sql -> use the standard db reader
• processor -> your own implementation of your simple logic
• writer to a file -> use the standard FlatFileItemWriter

second step

I don't undestand exactly what you mean by "process update and insert logic behind". I assume, that you read data from a db and based on that data, you have to execute inserts and updates in a table.

• reader for your more complex data -> again, use the standard db reader
• processor ->
  • prepare the string for the text file
  • prepare the new inserts and upates
• writer -> use a composite writer with the following delegates
  • FlatFileItemWriter for your textfile
  • DbWriter depending on your inserts and update needs

This way, you have clear transaction boundaries and can be sure, that the content of the file and inserts and updates are "in sync".

note: first and second step can run in parallel

third step - reader use a multiresource reader, to read from the two files - writer use a FlatFileItemWriter to write both contents into one file.

Of course, If you don't need to have the content in one file, then you can skip step 3.

You could also execute step 1 and 2 after each other and write in the same file. But depending on the execution time for step 1 and 2, the performance could be inferior to execute step 1 and 2 in parallel and using a third step to compine the data.

Answer 2

1. You can code a custom reader and write an application level logic in your custom processor for processing the inputs based on their content. It does not make sense to have two readers in one step. How would the spring batch execute them? It doesn't make sense to finish reader 1 and then start reader 2. This is as equal as having two different steps.

2. Another approach would be to place your output from both the reader in one file and then have another step for writing. But I'd go with the 1st technique.