Best timeout approach using HttpWebRequest.BeginGetResponse

Question

HttpWebRequest.BeginGetResponse does not respect any Timeout properties from the HttpWebRequest (Timeout or ReadWriteTimeout).

I read several approaches to get the same results, but I don’t know if this is the best way to do this, and if I will use for several calls, or I can scale it inside loops (I do webcrawler).

The important thing is that initially my code isn't async, I just need async because my method must accept a CancellationToken.

My concern is WaitHandles and ThreadPool.RegisterWaitForSingleObject. This is not a daily code, then I do not know if I can have problems in the near future.

private static void HandleCancellation(HttpWebRequest request, IAsyncResult getResponseResult, CancellationToken cancellationToken) { using (WaitHandle requestHandle = getResponseResult.AsyncWaitHandle) { ThreadPool.RegisterWaitForSingleObject(requestHandle, TimeoutCallback, request, request.Timeout, true); //If request finish or cancellation is called WaitHandle.WaitAny(new[] {requestHandle, cancellationToken.WaitHandle}); } //If cancellation was called if (cancellationToken.IsCancellationRequested) { request.Abort(); cancellationToken.ThrowIfCancellationRequested(); } } 

Challenge (again, this is not async)

 IAsyncResult getResponseResult = request.BeginGetResponse(null, null); HandleCancellation(request, getResponseResult, cancellationToken); return (HttpWebResponse)request.EndGetResponse(getResponseResult); 

Link: Best Approach to Managing Multiple WebRequest

Answer 1

The MSDN documentation for BeginGetResponse has a very good example of how to handle timeouts. This worked pretty well for me in my web crawler.