How does Zesek work with a huge payload?

Question

I read about Yesk at https://github.com/gresrun , and I would like to understand how it works under a huge payload. The only way to organize a job to create an instance of the Job class, and then use Thread to get started, or any other approaches? I am a little skeptical about using java.lang.Thread objects, as is done in the example at this link for batch jobs, where the data payload is huge.

Answer 1

In fact, killing threads without control is never a good idea.

I would suggest an approach to put your workers in a BlockingQueue and then spawn a very limited number of threads (as many as your processors to reduce competition) to get started with these workers. As soon as the work is completed, the flow will pick up a new employee and begin the process again. When there is no worker in the queue, threads simply hang in the queue, waiting for new workers.

You can see the thread pool template