Spring batch - Can we read items from database using date column in a loop

Question

I want to read data for last 30 days from database for each day..

ItemReader - Read data for each day ItemProcessor - Process data for each day ItemWriter - Write the processed data into database.

I want to repeat this process till date..

public boolean processTicketStatistics() {
        LocalDate startDate = LocalDate.now().minus(Period.ofDays(30));
        LocalDate endDate = LocalDate.now().plus(Period.ofDays(1));

        for (LocalDate d = startDate; d.isBefore(endDate); d = d.plusDays(1)) {
            TicketStatistics statistics = new TicketStatistics();
            statistics.setDate(localDateTimeToDate(d.atStartOfDay()));
            statistics.setTickets(ticketRepository.count(TicketSpecification.ticketHasDateRange(
                    localDateTimeToDate(d.atStartOfDay()), 
            localDateTimeToDate(d.atTime(LocalTime.MAX)))));
            ticketStatisticsRepository.save(statistics);
        }
        return true;
    }

Can you please help me how to achieve this in Spring batch?

I added an answer. If it helped, please accept it or upvote it: stackoverflow.com/help/someone-answers. Otherwise please let me know what is missing. Thank you. — Mahmoud Ben Hassine

Mahmoud Ben Hassine Mahmoud Ben Hassine · Accepted Answer · 2021-05-04T10:55:28

I would create a job that processes the data of a given day (ie the day is an identifying job parameter) and launch a job instance per day, for example:

LocalDate startDate = LocalDate.now().minus(Period.ofDays(30));
LocalDate endDate = LocalDate.now().plus(Period.ofDays(1));

for (LocalDate d = startDate; d.isBefore(endDate); d = d.plusDays(1)) {
    JobParameters jobParameters = new JobParametersBuilder()
            .addDate("date", toDate(d)) // TODO implement toDate to convert LocalDate to Date
            .toJobParameters();
    jobLauncher.run(job, jobParameters);
}

If the processing of each day is independent from other days (which I guess is the case), you can launch all job instances in parallel by configuring an asynchronous TaskExecutor (like a ThreadPoolTaskExecutor with a pool of 30 threads) in your JobLauncher.

Spring batch - Can we read items from database using date column in a loop

1 Answers