Java - Spring-Batch and Quartz and Tomcat - Returning cached instance of job bean

0
votes

I've been trying to setup a Spring-Batch job using Quartz, all running in a Tomcat container. I've got everything working using a number of Posts from Stackoverflow and various articles.

The batch job is setup to run every minute using Quartz and the CronExpression.

The issue now is that the actual Spring batch job only runs once. It's a successful run as the logs show output from both classes :

2016-03-09 11:20:00,047  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   RunFirstBatch.run  111 : RunFirstBatch().run() - Starting.................................[]
2016-03-09 11:20:00,047  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   RunFirstBatch.run  112 : RunFirstBatch().run() - CWD......................................[C:\Users\n0002501\AppData\Local\CI Eclipse for Java EE]

...removed for brevity...

2016-03-09 11:20:03,880  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   FirstBatch.execute  113 : FirstBatch().execute().................................................[** First Batch Job is Executing! **]
2016-03-09 11:20:03,880  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   FirstBatch.execute  114 : FirstBatch().execute() Step Contribution...............................[[StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]]
2016-03-09 11:20:03,880  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   FirstBatch.execute  115 : FirstBatch().execute() Chunk Context...................................[ChunkContext: attributes=[], complete=false, stepContext=SynchronizedAttributeAccessor: [], stepExecutionContext={batch.stepType=org.springframework.batch.core.step.tasklet.TaskletStep, batch.taskletType=com.lmig.batch.FirstBatch}, jobExecutionContext={}, jobParameters={}]

...removed for brevity...

2016-03-09 11:20:05,542  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   SimpleJobLauncher.run  136 : Job: [FlowJob: [name=firstBatchJob]] completed with the following parameters: [{}] and the following status: [COMPLETED]
2016-03-09 11:20:05,542  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   RunFirstBatch.run  126 : RunFirstBatch().run() - Exit Status..............................[COMPLETED]
2016-03-09 11:20:05,542  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1   RunFirstBatch.run  145 : RunFirstBatch().run()............................................[finally]

The Problem is on all sub-sequent runs, Spring returns a cached instance of the batch job class and doesn't run it stating that it's in a completed status : 

2016-03-09 11:21:00,008  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   RunFirstBatch.run  111 : RunFirstBatch().run() - Starting.................................[]
2016-03-09 11:21:00,008  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   RunFirstBatch.run  112 : RunFirstBatch().run() - CWD......................................[C:\Users\n0002501\AppData\Local\CI Eclipse for Java EE]

...removed for brevity...

...Returning cached instance of singleton bean 'firstBatchJob'

...removed for brevity...

2016-03-09 11:21:01,730  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   SimpleStepHandler.shouldStart  217 : Step already complete or not restartable, so no action to execute: StepExecution: id=1, version=3, name=stepOne, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
2016-03-09 11:21:01,730 DEBUG org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   SimpleFlow.resume  178 : Completed state=firstBatchJob.stepOne with status=COMPLETED
2016-03-09 11:21:01,730 DEBUG org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   SimpleFlow.resume  164 : Handling state=firstBatchJob.end1
2016-03-09 11:21:01,730 DEBUG org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   SimpleFlow.resume  178 : Completed state=firstBatchJob.end1 with status=COMPLETED
2016-03-09 11:21:01,730 DEBUG org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   AbstractJob.execute  305 : Job execution complete: JobExecution: id=1, version=1, startTime=Wed Mar 09 11:21:00 EST 2016, endTime=null, lastUpdated=Wed Mar 09 11:21:00 EST 2016, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, Job=[firstBatchJob]], jobParameters=[{}]

...removed for brevity...

2016-03-09 11:21:02,050  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   SimpleJobLauncher.run  136 : Job: [FlowJob: [name=firstBatchJob]] completed with the following parameters: [{}] and the following status: [COMPLETED]
2016-03-09 11:21:02,050  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   RunFirstBatch.run  126 : RunFirstBatch().run() - Exit Status..............................[COMPLETED]
2016-03-09 11:21:02,050  INFO org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2   RunFirstBatch.run  145 : RunFirstBatch().run()............................................[finally]

Things I've tried :

  • scope="prototype"
  • restartable="true"
  • implemented an ApplicationContextAware class to get the ApplicationContext.
  • implemented an JobExecutionListener class for beforeJob() and afterJob().

Here are the details : 

<bean id="batchApplicationContext" class="com.lmig.cm.rore.refmigrator.BatchApplicationContextProvider" scope="singleton"/>    

<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
    <property name="transactionManager" ref="transactionManager"/>
</bean>

<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher" scope="prototype">
    <property name="jobRepository" ref="jobRepository" />
</bean>

<bean id="firstBatch" class="com.lmig.batch.FirstBatch" scope="prototype"/>
<batch:step id="firstBatchStepOne">
    <batch:tasklet ref="firstBatch"/>
</batch:step>
<batch:job id="firstBatchJob" restartable="true">
    <batch:step id="stepOne" parent="firstBatchStepOne"/>
</batch:job>

<bean id="runFirstBatch" class="com.lmig.cm.rore.refmigrator.RunFirstBatch">
    <property name="context" ref="batchApplicationContext" />
</bean>

RunFirstBatch class : 

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;

public class RunFirstBatch {

    public static final String ID              = RunFirstBatch.class.getName ();
    private String             SHORT_NAME      = "RunFirstBatch()";
    @SuppressWarnings("unused")
    private String                  SYSTEM_IDENTITY = String.valueOf ( System.identityHashCode ( this ) );

    private     Log                 log = LogFactory.getLog(RunFirstBatch.class.getName());

    //private     ApplicationContext  context = null;
    private     BatchApplicationContextProvider context;

    /**
     * Default constructor.
     */
    public RunFirstBatch() {
        this.context = new BatchApplicationContextProvider();
    }

    public  void    run() { 

        try { 
            getLog().info ( SHORT_NAME + ".run() - Starting.................................[" + "]" );
            getLog().info ( SHORT_NAME + ".run() - CWD......................................[" + System.getProperty ( "user.dir" ) + "]" );

            //context = WebApplicationContextUtils.getWebApplicationContext(this.getServletContext());
            //this.context = new ClassPathXmlApplicationContext("/RoreWebConfiguration.xml");

            //String[] springConfig  = {"/first-batch.xml"};
            //context = new ClassPathXmlApplicationContext ( springConfig );

            if ( getContext() != null ) { 
                if ( getContext().getApplicationContext () != null ) { 
                    JobLauncher jobLauncher = (JobLauncher)getContext().getApplicationContext ().getBean("jobLauncher");
                    Job job = (Job) getContext().getApplicationContext ().getBean("firstBatchJob");
                    JobExecution execution = jobLauncher.run ( job, new JobParameters() );
                    //jobLauncher.run ( job, new JobParameters() );
                    getLog().info ( SHORT_NAME + ".run() - Exit Status..............................[" + execution.getStatus () + "]" );
                }
                else { 
                    getLog().info ( SHORT_NAME + ".run() - ApplicationContext is NULL...............[NULL]" );

                }
            }
            else { 
                getLog().info ( SHORT_NAME + ".run() - BatchApplicationContext is NULL..........[NULL]" );

            }

        }
        catch ( Exception ltheXcp ) { 
            getLog().error ( SHORT_NAME + ".run() - Exception ...............................[" + ltheXcp.getMessage () + "]" );
            getLog().error ( ltheXcp );

        }
        finally { 
            getLog().info ( SHORT_NAME + ".run()............................................[finally]" );
            //if (context != null) {
            //    context = null;
            //}

        }
    }

    public Log getLog() {
        return log;
    }

    public void setLog(Log log) {
        this.log = log;
    }

    public BatchApplicationContextProvider getContext() {
        return context;
    }

    public void setContext(BatchApplicationContextProvider context) {
        this.context = context;
    }


}

FirstBatch class : 

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;

import com.lmig.cm.rore.refmigrator.CMRoreReferenceMigrator;

public class FirstBatch implements Tasklet {

    public static final String ID              = FirstBatch.class.getName ();
    private String             SHORT_NAME      = "FirstBatch()";
    @SuppressWarnings("unused")
    private String              SYSTEM_IDENTITY = String.valueOf ( System.identityHashCode ( this ) );

    private     Log             log = LogFactory.getLog(FirstBatch.class.getName());

    public FirstBatch() {
        // TODO Auto-generated constructor stub
    }

    public FirstBatch( Log theLog ) {
        this.log = theLog;
    }

    /* (non-Javadoc)
     * @see org.springframework.batch.core.step.tasklet.Tasklet#execute(org.springframework.batch.core.StepContribution, org.springframework.batch.core.scope.context.ChunkContext)
     */
    @Override
    public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
        throws Exception {
        getLog().info(SHORT_NAME + ".execute().................................................[** First Batch Job is Executing! **]");
        getLog().info(SHORT_NAME + ".execute() Step Contribution...............................[" + arg0.toString () + "]");
        getLog().info(SHORT_NAME + ".execute() Chunk Context...................................[" + arg1.toString () + "]");
        return RepeatStatus.FINISHED;
        //return RepeatStatus.CONTINUABLE;

    }

    public Log getLog() {
        return log;
    }

    public void setLog(Log log) {
        this.log = log;
    }


}

I suspect it's something simple, but I can't seem to find the correct setting, attribute or option to cause it to start the job fresh again.

thx in advance, adym

The solution per Michael's answer :

  • Spring Batch 3.0.2
  • JDK 1.7-79
  • Spring 3.2.14
  • Quartz 2.2.2
  • Tomcat 6.0.44

The solution was to simply provide at least 1 JobParameter in the JobParameters() that allows Spring-Batch to "uniquely" identify one JobInstance versus another. I just used the current Date/Time as a String :

Add the following code in the RunFirstBatch.run() method : 

        SimpleDateFormat    theFormat = new SimpleDateFormat ( "yyyy-MM-dd-HH-mm-ss" );

        if ( getBatchContext ().getApplicationContext () != null ) { 
            if ( getBatchContext ().getApplicationContext () != null ) { 
                JobLauncher     jobLauncher = (JobLauncher)getBatchContext ().getApplicationContext ().getBean("jobLauncher");
                Job             job = (Job) getBatchContext ().getApplicationContext ().getBean("firstBatchJob");

                // JOBPARAMETERS : Build the job parameters...
                // 
                Date            theDate = new Date();
                // format the date as yyyy-MM-dd-HH-mm-ss 
                String          theJobId = theFormat.format ( theDate );
                // job parameter (single) from the formatted date...
                JobParameter    idParm = new JobParameter ( theJobId );
                // parameters container...used for the JobParameters
                // constructor...
                Map<String,JobParameter> mapParms = new HashMap<String,JobParameter> ();
                mapParms.put ( "1", idParm );

                JobParameters   theParms = new JobParameters (mapParms);

                getLog().info ( SHORT_NAME + ".run() - Job Id...................................[" + theJobId + "]" );


                JobExecution    execution = jobLauncher.run ( job, theParms );
                //execution.setExitStatus ( ExitStatus.UNKNOWN );
                //jobLauncher.run ( job, new JobParameters() );
                getLog().info ( SHORT_NAME + ".run() - Exit Status..............................[" + execution.getStatus () + "]" );
            }
            else { 
                getLog().info ( SHORT_NAME + ".run() - ApplicationContext is NULL...............[NULL]" );

            }
        }
        else { 
            getLog().info ( SHORT_NAME + ".run() - BatchApplicationContext is NULL..........[NULL]" );

        }
1
javaspringspring-batch
So, if I set the FirstBatch.execute() return status to return RepeatStatus.CONTINUABLE;, the job runs everytime, but just keeps running continuously. But I need it to stop after each run. - lincolnadym

1 Answers

1
votes

keep the RepeatStatus.FINISHED but look into https://docs.spring.io/spring-batch/reference/html/domain.html#domainJobParameters

"how is one JobInstance distinguished from another?" The answer is: JobParameters. JobParameters is a set of parameters used to start a batch job.

and

Thus, the contract can be defined as: JobInstance = Job + identifying JobParameters. This allows a developer to effectively control how a JobInstance is defined, since they control what parameters are passed in.

... so think about a identifying jobparameter (random, unique, maybe Date.getTime())?