Limit the number of threads in Groovy

Question

How can I limit the number of threads that execute at one time? Here is an example of my algorithm:

for(i = 0; i < 100000; i++) { Thread.start { // Do some work } } 

I would like to make sure that when the number of threads in my application reaches 100, the algorithm will stop / wait until the number of threads in the application drops below 100.

Currently, "some work" takes some time, and in the end I get several thousand threads in my application. In the end, he runs out of threads, and "some work" fails. I would like to fix this by limiting the number of pools that it can use at one time.

Please let me know how to solve my problem.

Answer 1

I believe that you are looking for ThreadPoolExecutor in the Java Concurrency API. The idea here is that you can determine the maximum number of threads in the pool, and then instead of starting new threads using Runnable, just let ThreadPoolExecutor take care of managing the upper limit for threads.

Start here: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html

 import java.util.concurrent.*; import java.util.*; def queue = new ArrayBlockingQueue<Runnable>( 50000 ) def tPool = new ThreadPoolExecutor(5, 500, 20, TimeUnit.SECONDS, queue); for(i = 0; i < 5000; i++) { tPool.execute { println "Blah" } } 

ThreadBlockingQueue constructor parameters: corePoolSize (5) , this is the number of threads to create and maintain, if the system is inactive, maxPoolSize (500) maximum number of threads to create, the third and fourth arguments indicate that the pool should support idle threads for at least 20 seconds, and the queue argument is a blocking queue that stores tasks in the queue.

What you need to play with is the size of the queue, as well as how to handle rejected tasks. If you need to complete 100k tasks, you will either have to have a queue that can contain 100k tasks, or you will have to have a strategy to handle rejected tasks.