Spring Batch - How to read one large file in multiple threads?

Question

Problem: Read a file> 10 MB in size and upload it to the staging table using Spring Batch. How can we maintain state while reading a file to restart the task if it fails?

According to the documentation, FileItemReader is not thread safe, and if we try to make it thread safe, we will lose the restart. So, the main questions:

• Is there a way to read a file in blocks, and each thread knows which block to read?
• If we make the reading synchronous, what changes are needed to restart the job in this scenario?

If someone has encountered similar problems or has any analysis of how it works, we can make a decision.

Any pointers or example codes are also evaluated.

Answer 1

Multithreading is only useful if your threads do different things at the same time. For example, you can have two threads running on separate processors. Or one thread can wait for a network message while another draws a screen.

But in your case, both threads will wait for the same I / O from the same device, so it makes no sense to use more than one.

See also this question. Reading a file with multiple threads.