How to Plan a Spring Bootable Batch Application

Question

I have a Spring boot application:

import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class LoadApplication { public static void main(String[] args) { SpringApplication.run(LoadApplication.class, args); } } 

In the @Configuration class, also annotated with @EnableBatchProcessing , I have the following bean batch job:

 @Bean public Job loadJob(JobBuilderFactory jobs, Step stepLoadFile, Step stepArchiveFile) { return jobs.get("loadJob") .incrementer(new RunIdIncrementer()) .start(stepLoadFile) .next(stepArchiveFile) .build(); } 

The stepLoadFile job stepLoadFile reads a flat file (see below) and writes the file data to the database. stepArchiveFile then moves the file to another directory.

Typically, work should be done once a day (Tuesday - Saturday) at the indicated time. However, if the flat file is not found, the task does not work, and it must be restarted every 30 minutes until it is completed or the maximum number of attempts is reached (for example, 5). After a successful restart, the job should not start again until the next normal run time. In addition, the system should ideally prevent parallel runs of the same job. How can all this be done?

Note. Repeat playback is not required to get where the previous job launch ended. This is because the block size is larger than the number of elements in the file.

I tried this in my @Configuration class (Note: I also added @EnableRetry to the configuration and main class):

 @Bean public ItemReader<Test> reader(LineMapper<Test> lineMapper, ApplicationProperties properties) { FlatFileItemReader<Test> flatFileItemReader = new FlatFileItemReader<Test>() { @Override @Retryable(value = {ItemStreamException.class}, maxAttempts=5) public void open(ExecutionContext executionContext) throws ItemStreamException { super.open(executionContext); } @Override @Retryable(maxAttempts=5) public Holding read() throws UnexpectedInputException, ParseException, Exception { return super.read(); } }; flatFileItemReader.setResource(new FileSystemResource(properties.getLoadFile())); flatFileItemReader.setLineMapper(lineMapper); return flatFileItemReader; } 

ItemStreamException thrown and the application exits without ItemStreamException .

Answer 1

You can plan by adding the following component to the LoadApplication class

 @Component @EnableScheduling class ScheduledTasks { @Autowired JobLauncher jobLauncher; @Autowired Job job; /**The pattern is a list of six single space-separated fields: * representing second, minute, hour, day, month, weekday. * Month and weekday names can be given as the first three letters of the English names. * Example patterns: "0 0 * * * *" = the top of every hour of every day.* "*\/10 * * * * *" = every ten seconds. Remove 2nd character, it is escape "0 0 8-10 * * *" = 8, 9 and 10 o'clock of every day. "0 0/30 8-10 * * *" = 8:00, 8:30, 9:00, 9:30 and 10 o'clock every day. "0 0 9-17 * * MON-FRI" = on the hour nine-to-five weekdays "0 0 0 25 12 ?" = every Christmas Day at midnight */ @Scheduled(cron = "0 0/30 * * * TUE-SAT") public void runJob() throws Exception { jobLauncher.run(job, new JobParameters()); } } 

To retry unsuccessful steps, you may need to configure it at the work stage.

<chunk reader="itemReader" writer="itemWriter" commit-interval="2" retry-limit="5"> <retryable-exception-classes> <include class="java.io.FileNotFoundException"/> </retryable-exception-classes> </chunk>

Also, if you store the spring-batch metadata tables on disk and not in memory, the spring package will not start the same job again.

Answer 2

Use conditional stream in step declaration.
<step id="readWrite"> <tasklet>(...)</tasklet> <next on="FAILED" to="scheduleNext" /> </step> <step id="scheduleNext"> <tasklet ref="schedulerTasklet"> </step> <bean id="schedulerTasklet" class="SchedulerTasklet"/>

and in the following SchedulerTasklet schedule:

 public class SchedulerTasklet implements Tasklet { @Autowired private TaskScheduler taskScheduler; public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception { Runnable task = new Runnable() { public void run() { run your job here } }; taskScheduler.schedule(task, delay); return RepeatStatus.FINISHED; } }