Spring batch chunk based approach for calling REST API in Item Writer

Question

I have a requirement to read data from our RDBMS table and invoke a REST API in order to ingest the information into downstream system .

Downstream REST API only allows a single item to be passed at a time.

Is chunk based spring batch a better option for this requirement as I am not using the power of Item writer processing all items at a time?

Usually when we use Item writers like JPA ItemWriter ,item writer offers advantage of storing entire chunk of data in a single transaction through data base batch operations.

But that advantage is not possible when target I/O is REST API.

I can run chunks in parallel.

Would there still be other advantages of using spring batch in this requirement rather than a custom implementation of reading from database and invoking API?

Answer 1

Downstream REST API only allows a single item to be passed at a time.

You could probably use a chunk-oriented step with a chunkSize=1. This is usually not a good idea as there will be a transaction for each item (no buffering and bulk-writes as you mentioned), but it might work/perform well for small/medium datasets.

Would there still be other advantages of using spring batch in this requirement rather than a custom implementation of reading from database and invoking API?

Yes, restartability. If your job fails at some point, Spring Batch allows you to restart it from where it left off (ie do not re-process items that have been read and posted to your REST endpoint in the previous run). If you go the custom implementation route, you would need to implement this yourself (unless you don't care about re-reprocessing everything twice).