Spring batch - In memory summary of input records before writing the output

Question

We need to read a large amount of data (a few millions of records), do some business logic on it and write the results once.

E.g.

Input

Ringo, Elephant, London Zoo

Zooshu, Horse, In the Wild

Domo, Panda, Singapore Farm Animal Sanctuary

Sasha, Elephant, Bombai zoological garden

Output

• 2 animals in Asia
• 3 unique species
• 3 animals in captivity
• ...

We are wondering if Spring Batch can be a good platform for this because as we understand it, every ItemReader flows eventually to an Item Writer and we only need to write once.

Answer 1

I think Spring Batch is a good fit in this scenario. ItemReader will read from the input file and pass every line to a ItemProcessor, where your business logic resides. After processing, ItemWriter can writer out the result to a file. Go ahead and create a toy with it. https://spring.io/guides/gs/batch-processing/