Spring Batch, read whole csv file before reading line by line

1
votes

I want to read a csv file, enrich each row with some data from some other external system and then write the new enriched csv to some directory Now to get the data from external system i need to pass each row one by one and get the new columns from external system. But to query the external system with each row i need to pass a value which i have got from external system by sending all the values of a perticular column. e.g - my csv file is -

name, value, age

10,v1,12

11,v2,13

so to enrich that i first need to fetch a value as per total age - i.e 12 + 13 and get the value total from external system and then i need to send that total with each row to external system to get the enriched value. I am doing it using spring batch but using fLatFileReader i can read only one line at a time. How would i refer to whole column before that.

Please help.

Thanks

1
springspring-batch

1 Answers

2
votes

There are two ways to do this.

OPTION 1

Go for this option if you are okey to store all the records in memory. Totally depends how many record you need to calculate the total age.

Reader(Custom Reader) : Write the logic to read one line at a time. You need to return null from read() only when you feel all the lines are read for calculating the total age.

NOTE:- A reader will loop the read() method until it returns null.

Processor : You will get the full list of records. calculate the total age. Connect the external system and get the value. Form the records which need to be written and return from the process method.

NOTE:- You can return all the records modified by a particular field or merge a single record. This is totally your choice what you would like to do.

Writer : Write the records.

OPTION 2

Go for this if option1 is not feasible.

Step1: read all the lines and calculate the total age and pass the value to the next step.

Step2: read all the lines again and update the records with required update and write the same.