Spring batch & job scheduling

Question

Our team currently runs a number of scheduled jobs which we've inherited from past teams, it's not a lot of jobs but enough to make it difficult to keep track of. The main problems we're facing with our current setup are: - No clear visibility about what runs and when - Different ways of notifying job failures - Not all jobs support re-running, pot luck as to whether re-running will work or make things worse.

To that end we've been looking at re-writing our jobs using Spring Batch and using SOS Job Scheduler to manage them.

Has anyone else used this approach or can recommend a good approach for managing jobs?

Answer 1

I'm not sure you have a real question here, but yes, jobs written in Spring Batch are great with a job scheduler. I've written dozens of jobs scheduled through AutoSys and Tivoli; not familiar with SOS, but I expect it's pretty much the same.

The main trick in scheduling jobs via Spring Batch is in determining the parameters your job needs.

Spring Batch uses the concepts of JobInstance and JobExecution to manage jobs; an instance is defined by a job name and associated parameters, while an execution is an attempt at running an instance. Once an execution has run to completion, the instance is also complete and cannot be rerun.

Practically, that means if a job doesn't use outside parameters (i.e., most recurring scheduled jobs) it will need some sort of synthetic parameter passed in at every run, because if you don't pass something in as a parameter, you'll only ever be able to run it once (because that empty set of parameters defines a unique JobInstance).

If the job is run frequently, I've used something like the current timestamp, which insures every JobInstance is unique. If it runs daily, passing in the current date ensures you get one JobInstance per day, which is good for tracking and re-starting.

All in all, scheduled Spring Batch jobs are not complicated, and work well.

Edit in response to questioner's comment:

Something that didn't make sense to me about Spring Batch's model is that the logic for determining job parameters is all externalized; that might be OK in a lot of situations where parameters like a date can easily be generated by a calling script, but I had a lot of jobs that had more complexity to the parameter logic.

What I ended up doing was to extend the CommandLineJobRunner to take an instance of a new interface named something like JobParameterGenerator. Implementations ofJobParameterGenerator were generally specific to the particular job; for example, one implementation had to do some database lookups to formulate an appropriate date range for the job. I also created a basic JobParameterGenerator that did nothing but return the current DateTime for use in the jobs that were always new instances. Hope this helps.