Spring Batch Item Reader runs only once

Trying to implement Spring package, but faced with some strange problem, our ItemReaderclass is executed only once.

Details are provided below.

  • If we have 1000 rows in the database.
  • The reader of our products extracts 1000 rows from the database and passes the list ItemWriter
  • ItemWriter successfully delete all items.
  • Now ItemReader again tries to retrieve data from the database, but did not find it, therefore returns NULL, therefore, execution stops.
  • But we set up a package that will run with a scheduler Quartzthat runs every minute.
  • Now, if we insert let 1000 rows into the DB using the dump import, the batch job should select this data in the next execution, but it will not even be executed, although it will be executed JobLauncher.

Configuration: -

1. We have an ItemReader, ItemWriter with a commit interval of 1.

<batch:job id="csrfTokenBatchJob">
    <batch:step id="step1">
      <tasklet>
        <chunk reader="csrfTokenReader" writer="csrfTokenWriter" commit-interval="1"></chunk>
      </tasklet>
    </batch:step>
  </batch:job>

2. Scheduled to run every minute.

<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
    <property name="triggers">
      <bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
        <property name="jobDetail" ref="jobDetail" />
        <property name="cronExpression" value="0 0/1 * * * ?" />
      </bean>
    </property>
  </bean>

3.Job configuration

<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
    <property name="jobClass" value="com.tavant.oauth.batch.job.CSRFTokenJobLauncher" />
    <property name="jobDataAsMap">
        <map>
            <entry key="jobName" value="csrfTokenCleanUpBatchJob" />
            <entry key="jobLocator" value-ref="jobRegistry" />
            <entry key="jobLauncher" value-ref="jobLauncher" />
        </map>
    </property>
</bean>

The first time it succeeds, but later it fails, but I can see in the logs being executed JobLauncher.

@Component("csrfTokenReader")
@Scope(value="step")
public class CSRFTokenReader implements ItemReader<List<CSRFToken>> {

    private static final Logger logger = LoggerFactory.getLogger(CSRFTokenReader.class);

    @Autowired
    private CleanService cleanService;

    @Override
    public List<CSRFToken> read() {
        List<CSRFToken> csrfTokenList = null;
        try{

            int keepUpto = Integer.valueOf(PropertiesContext.getInstance().getProperties().getProperty("token.keep", "1"));

            Calendar calTime = Calendar.getInstance();
            calTime.add(Calendar.HOUR, -keepUpto);
            Date toKeep = calTime.getTime();

            csrfTokenList = cleanService.getCSRFTokenByTime(toKeep);
        }
        catch(Throwable th){
            logger.error("Exception in running job At " + new Date() + th);
        }
        if(CollectionUtils.isEmpty(csrfTokenList)){
            return null;
        }
        return csrfTokenList;
    }
}

EDIT: -

public class CSRFTokenJobLauncher extends QuartzJobBean {
    static final String JOB_NAME = "jobName";
    private JobLocator jobLocator;
    private JobLauncher jobLauncher;
    public void setJobLocator(JobLocator jobLocator) {
        this.jobLocator = jobLocator;
    }
    public void setJobLauncher(JobLauncher jobLauncher) {
        this.jobLauncher = jobLauncher;
    }
    @Override
    protected void executeInternal(JobExecutionContext context) {
        Map<String, Object> jobDataMap = context.getMergedJobDataMap();
        String jobName = (String) jobDataMap.get(JOB_NAME);
        log.info("Quartz trigger firing with Spring Batch jobName="+jobName);
        JobParameters jobParameters = getJobParametersFromJobMap(jobDataMap);
        try {
            jobLauncher.run(jobLocator.getJob(jobName), jobParameters);
        }
        catch (JobExecutionException e) {
            log.error("Could not execute job.", e);
        }
    }
    private JobParameters getJobParametersFromJobMap(Map<String, Object> jobDataMap) {
        JobParametersBuilder builder = new JobParametersBuilder();
        for (Entry<String, Object> entry : jobDataMap.entrySet()) {
            String key = entry.getKey();
            Object value = entry.getValue();
            if (value instanceof String && !key.equals(JOB_NAME)) {
                builder.addString(key, (String) value);
            }
            else if (value instanceof Float || value instanceof Double) {
                builder.addDouble(key, ((Number) value).doubleValue());
            }
            else if (value instanceof Integer || value instanceof Long) {
                builder.addLong(key, ((Number)value).longValue());
            }
            else if (value instanceof Date) {
                builder.addDate(key, (Date) value);
            }
        }
        return builder.toJobParameters();
    }
}
+4
source share
2 answers

After hours of wasting time, the problem seems to be resolved now, I configured allow-start-if-complete="true"in tasklet.Now Batch Item Reader runs according to schedule.

<batch:job id="csrfTokenBatchJob">
    <batch:step id="step1">
      <batch:tasklet allow-start-if-complete="true">
        <batch:chunk reader="csrfTokenReader" writer="csrfTokenWriter" commit-interval="1"></batch:chunk>
      </batch:tasklet>
    </batch:step>
  </batch:job>
+4
source

Spring batch . spring . , , , , .

OPTION1: - , allow-start-if-complete = "true"

OPTION2: - , . .

    JobExecution jobExecution = jobLauncher.run(reportJob, new JobParametersBuilder()
                    .addDate("now", new Date())

OPTION3: - , RunIdIncrementer, .

    @Bean
    public Job job1(JobBuilderFactory jobs, Step s1) {
        return jobs.get("job1")
                .incrementer(new RunIdIncrementer())
                .flow(s1)
                .end()
                .build();
    }
+1

All Articles