We have several long business processes that run through WCF services running in IIS (integrated mode) on WS 2008 R2. These business processes typically involve a lot of interaction with our SQL Server. We have created the usual implementation of a task queue by which requests are queued through the initial call to a service and then executed based on priority. This execution can take a lot of time (maximum 20-30 minutes). Clients can then request a server to perform their own background tasks.
In our current implementation, tasks are launched in a separate thread for execution, and not from ThreadPool. This was due to reading recommendations not to run lengthy tasks using ThreadPool to prevent ASP.NET requests from starving from serving. We control the number of threads generated by setting an upper limit on the number of background tasks that can be performed simultaneously. In this way, we are trying to control the load on the CPU and prevent too many thread context switches. Although all this happens, we, of course, should still handle the usual "on-line" requests for the application.
After reading this post by Thomas Marquardt, I am concerned about the fact that we are not using ThreadPool, since we will not get the advantage of the heuristic settings built into it. We are already solving the shutdown problem that Thomas mentions by connecting to the ApplicationEnd event and canceling lengthy tasks. So my question is: are we moving on to using ThreadPool? How about these streams being connected over long periods of time? If I understand Thomas correctly, he says that it doesn’t matter, since ThreadPool will set up to create more requests to serve ordinary online operations? I have also read https://stackoverflow.com/a/166148/ which covers the same reasons, but I'm still not sure about what to do next.