Why is blocking canceled so long when many HTTP requests are canceled?

Question

Background

I have code that performs batch processing of HTML pages using content from one specific host. He is trying to make a large number (~ 400) of concurrent HTTP requests using HttpClient . I believe that the maximum number of concurrent connections is limited by ServicePointManager.DefaultConnectionLimit , so I do not apply my own concurrency limits.

After sending all requests asynchronously to HttpClient using Task.WhenAll , the entire batch operation can be canceled using CancellationTokenSource and CancellationToken . Operation progress can be viewed using the user interface, and the button can be pressed to cancel.

Problem

The CancellationTokenSource.Cancel() call blocks for about 5-30 seconds. This causes the user interface to freeze. This is suspected to occur because the method calls the code registered for the failure notice.

What i reviewed

• Limit the number of concurrent HTTP request requests. I find this to be workaround because HttpClient already seems like it is loading extra requests itself.
• Making a call to the CancellationTokenSource.Cancel() method on a thread other than the UI. This did not work too well; the task was not actually carried out until most of the others had ended. I think the version of the async method will work well, but I could not find it. In addition, I get the impression that it is suitable for using a method in a user interface thread.

Demonstration

Code

 class Program { private const int desiredNumberOfConnections = 418; static void Main(string[] args) { ManyHttpRequestsTest().Wait(); Console.WriteLine("Finished."); Console.ReadKey(); } private static async Task ManyHttpRequestsTest() { using (var client = new HttpClient()) using (var cancellationTokenSource = new CancellationTokenSource()) { var requestsCompleted = 0; using (var allRequestsStarted = new CountdownEvent(desiredNumberOfConnections)) { Action reportRequestStarted = () => allRequestsStarted.Signal(); Action reportRequestCompleted = () => Interlocked.Increment(ref requestsCompleted); Func<int, Task> getHttpResponse = index => GetHttpResponse(client, cancellationTokenSource.Token, reportRequestStarted, reportRequestCompleted); var httpRequestTasks = Enumerable.Range(0, desiredNumberOfConnections).Select(getHttpResponse); Console.WriteLine("HTTP requests batch being initiated"); var httpRequestsTask = Task.WhenAll(httpRequestTasks); Console.WriteLine("Starting {0} requests (simultaneous connection limit of {1})", desiredNumberOfConnections, ServicePointManager.DefaultConnectionLimit); allRequestsStarted.Wait(); Cancel(cancellationTokenSource); await WaitForRequestsToFinish(httpRequestsTask); } Console.WriteLine("{0} HTTP requests were completed", requestsCompleted); } } private static void Cancel(CancellationTokenSource cancellationTokenSource) { Console.Write("Cancelling..."); var stopwatch = Stopwatch.StartNew(); cancellationTokenSource.Cancel(); stopwatch.Stop(); Console.WriteLine("took {0} seconds", stopwatch.Elapsed.TotalSeconds); } private static async Task WaitForRequestsToFinish(Task httpRequestsTask) { Console.WriteLine("Waiting for HTTP requests to finish"); try { await httpRequestsTask; } catch (OperationCanceledException) { Console.WriteLine("HTTP requests were cancelled"); } } private static async Task GetHttpResponse(HttpClient client, CancellationToken cancellationToken, Action reportStarted, Action reportFinished) { var getResponse = client.GetAsync("http://www.google.com", cancellationToken); reportStarted(); using (var response = await getResponse) response.EnsureSuccessStatusCode(); reportFinished(); } } 

Output

Why does the cancellation unit last so long? Also, is there something I am doing wrong or could be better?

Answer 1

Making a call to the CancellationTokenSource.Cancel () method on a thread other than the UI. This did not work too well; the task was not actually carried out until most of the others had finished.

This suggests that you are probably suffering from a "thread depletion" in which your threadpool queue has so many elements in it (from HTTP requests terminates) that it takes some time to go through all of them. Undo probably blocks the execution of a particular threadpool work item, and it cannot go to the queue header.

This suggests that you need to go with option 1 from your review list. Throttle your own work so that the thread queue stays relatively short. In any case, it is well suited for responsive applications.

My favorite way to throttle asynchronous operation is to use Dataflow . Something like that:

 var block = new ActionBlock<Uri>( async uri => { var httpClient = new HttpClient(); // HttpClient isn't thread-safe, so protect against concurrency by using a dedicated instance for each request. var result = await httpClient.GetAsync(uri); // do more stuff with result. }, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 20, CancellationToken = cancellationToken }); for (int i = 0; i < 1000; i++) block.Post(new Uri("http://www.server.com/req" + i)); block.Complete(); await block.Completion; // waits until everything is done or canceled. 

Alternatively, you can use Task.Factory.StartNew passing in TaskCreationOptions.LongRunning so that your task receives a new thread (not associated with threadpool) that would allow it to start immediately and call Cancel from there. But instead, you should probably solve the problem of running out of thread.