Maximum Number of Items Delivered in ThreadPool.QueueUserWorkItem

Question

I set the maximum thread to 10. Then I added a 22000 task using ThreadPool.QueueUserWorkItem. It is very likely that not all 22,000 tasks were completed after the program was launched. Is there a limit on how many tasks can be queued for available threads?

Answer 1

There are no practical restrictions in the queue, however, the pool itself will not exceed 64 wait descriptors, i.e. shared threads are active.

Answer 2

If you need to wait until all tasks are processed, you will have to process it yourself. ThreadPool threads are background threads and will not keep the application operational.

This is a relatively clean way to handle this situation:

using (var mre = new ManualResetEvent(false)) { int remainingToProcess = workItems.Count(); // Assuming workItems is a collection of "tasks" foreach(var item in workItems) { // Delegate closure (in C# 4 and earlier) below will // capture a reference to 'item', resulting in // the incorrect item sent to ProcessTask each iteration. Use a local copy // of the 'item' variable instead. // C# 5/VS2012 will not require the local here. var localItem = item; ThreadPool.QueueUserWorkItem(delegate { // Replace this with your "work" ProcessTask(localItem); // This will (safely) decrement the remaining count, and allow the main thread to continue when we're done if (Interlocked.Decrement(ref remainingToProcess) == 0) mre.Set(); }); } mre.WaitOne(); } 

---

As they say, it is usually better to group your work items if you have thousands of them and not consider them as separate work items for threadpool. These are some of the overhead of managing a list of items, and since you won’t be able to process 22,000 at a time, you'd better group them into blocks. With separate work items, each 50 or so process is likely to help your overall throughput quite a bit ...

Answer 3

From ThreadPool documentation :

Note. . Threads in a managed thread pool are background threads. That is, their IsBackground properties are correct. This means that the ThreadPool thread will not support starting the application after all front threads exit.

Is it possible that you exit before all tasks are processed?

Answer 4

This is an implementation dependent question, and the implementation of this function has changed over time. But in .Net 4.0 you are significantly limited by the amount of memory in the system, since tasks are stored in a queue in memory. You can see this by digging an implementation in a reflector.