Spring Batch : Read multiple json files and process one by one

Question

I have huge JSON files in a folder approx 200 each of 300 mbs approx. I need to write Spring batch to read the files and process one by one.

Basically means read one file at a time then run the processor then run the writer then read next file, The file is basically customer data and activity dump, with each file containing several users data and many files. Like below

[{
"cutomerId":123,
"customerName":"bla babha",
"phoneNumber" : 5060974321,
..
,,
},{
"cutomerId":123,
"customerName":"bla babha",
"phoneNumber" : 5060974321,
..
..
}
]

Since I cant read all the files at one due to space issue.

Is this possible. what would be the configuration.

Please help.

docs.spring.io/spring-batch/trunk/apidocs/org/springframework/… — Luca Basso Ricci
Hi @LucaBassoRicci, Thanks for the reply , I read this post before but that doesn't seem to help. Since the configurations there are incomplete. Also I read through several MultiResourceItemReader but none of them show how to read json and secondly json with List of Object in this case Customer POJO. Also I am fairly new to Spring boot not able to make much out of it. — Lucie
Check this example :github.com/bigzidane/spring-batch-jsonListItem-reader — soorapadman

Mahmoud Ben Hassine Mahmoud Ben Hassine · Accepted Answer · 2019-03-22T14:40:51

Spring Batch v4.1 introduced the JsonItemReader which you can use to read your json files. More details about Json support can be found in the JSON Item Readers And Writers section of the reference documentation.

This reader can be used as a delegate to the MultiResourceItemReader (as Luca Basso Ricci mentioned in the comment) to read multiple json files in a directory. You can find an example here. The example shows a FlatFileItemReader as a delegate but you can use a similar configuration with a JsonItemReader instead.

Hope this helps.

Spring Batch : Read multiple json files and process one by one

1 Answers