HttpWebRequest is very slow!

Question

I am using an open source library to connect to my web server. I was worried that the web server was very slow, and then I tried to do a simple test in Ruby, and I got these results

Ruby program: 2.11 seconds for 10 HTTP GETs

Ruby program: 18.13 seconds for 100 HTTP GETs

C # library: 20.81 seconds for 10 HTTP GETs

C # library: 36847.46 seconds for 100 HTTP GETs

I profiled and found the problem with this function:

private HttpWebResponse GetRawResponse(HttpWebRequest request) { HttpWebResponse raw = null; try { raw = (HttpWebResponse)request.GetResponse(); //This line! } catch (WebException ex) { if (ex.Response is HttpWebResponse) { raw = ex.Response as HttpWebResponse; } } return raw; } 

The marked line takes more than 1 second to complete on its own, while the ruby ​​program executing 1 request takes 0.3 seconds. I also do all of these tests at 127.0.0.1, so network bandwidth is not a problem.

What could be causing this huge slowdown?

UPDATE

Check out the modified test results. I really tested 10 GET, not 100, I updated the results.

Answer 1

What I consider to be the main culprit for slow web requests is the proxy property. If you set this property to null before calling the GetResponse method, the request will skip the proxy auto-detection step:

 request.Proxy = null; using (var response = (HttpWebResponse)request.GetResponse()) { } 

Auto-detection of a proxy server took up to 7 seconds for a request before returning a response. It's a little annoying that this property is set by default for the HttpWebRequest object.

Answer 2

Perhaps this is due to the fact that you open several connections at once. By default, the maximum number of open HTTP connections is set to two. Try adding this to your .config file and see if it helps:

 <system.net> ....... <connectionManagement> <add address="*" maxconnection="20"/> </connectionManagement> </system.net> 

Answer 3

I had a similar problem with the VB.Net MVC project.
Locally on my computer (Windows 7), it took less than 1 second to search for pages, but on the server (Windows Server 2008 R2) it took 20 seconds for each page request.

I tried a combination of setting proxy to null

  System.Net.WebRequest.DefaultWebProxy = Nothing request.Proxy = System.Net.WebRequest.DefaultWebProxy 

And changing the configuration file, adding

  <system.net> ....... <connectionManagement> <add address="*" maxconnection="20"/> </connectionManagement> </system.net> 

This still did not reduce the slow page request time on the server. In the end, the solution was to uncheck the "Automatically detect settings" checkbox in the IE settings on the server itself. (In the "Tools → Internet Options" section, select the "Connections" tab. Click the "LAN Settings" button)

Immediately after I unchecked this browser option on the server, all page request times fell from 20 + seconds to less than 1 second.

Answer 4

I began to notice a slow, similar to OP in this area, which improved slightly with the increase in MaxConnections.

 ServicePointManager.DefaultConnectionLimit = 4; 

But after creating this amount of WebRequests, the delays returned.

The problem, in my case, was that I called POST and did not worry about the answer, so I did not intend to do anything with it. Unfortunately, this left WebRequest floating until they ran out.

The fix was to pick up the answer and just close it.

 WebRequest webRequest = WebRequest.Create(sURL); webRequest.Method = "POST"; webRequest.ContentLength = byteDataGZ.Length; webRequest.Proxy = null; using (var requestStream = webRequest.GetRequestStream()) { requestStream.WriteTimeout = 500; requestStream.Write(byteDataGZ, 0, byteDataGZ.Length); requestStream.Close(); } // Get the response so that we don't leave this request hanging around WebResponse response = webRequest.GetResponse(); response.Close(); 

Answer 5

Use a computer other than localhost, and then use WireShark to see what really happens through the wire.

Like others, this can be a few things. A look at things at the TCP level should give a clear picture.

Answer 6

I don’t know exactly how I got to this workaround, I didn’t have time to do any research, so it depends on you guys. There is a parameter, and I used it like this (in the constructor of my class, before creating the HTTPWebRequest object):

 System.Net.ServicePointManager.Expect100Continue = false; 

I don’t know why, but now my challenges look pretty fast.

Answer 7

I know this is some kind of old thread, but I lost all day with a slow HttpWebRequest, tried every provided solution with no luck. Each request to any address was more than one minute.

In the end, the problem was with my antivirus firewall (Eset) . I use an interactive mode firewall, but Eset is completely disabled . This prompted a request to last forever. After turning on Eset and executing the request, the message of the quick access firewall is displayed and after confirmation the request is executed in less than one second .

Answer 8

I tried all the solutions described here with no luck, the call took about 5 minutes.

What was the problem: I needed the same session and obviously the same cookies (a request made on the same server), so I restored the cookies from Request.Cookies to WebRequest.CookieContainer. The response time was about 5 minutes.

My solution: Commented on the cookie-related code and bam! The call took less than one second.

Answer 9

This worked for me:

 <configuration> <system.net> <defaultProxy enabled="false"/> </system.net> </configuration> 

Credit : Slow HTTPWebRequest when you first start the program

Answer 10

The problem for me was that I installed LogMeIn Hamachi - ironically, to remotely debug the same program, which then began to demonstrate this extreme slowness.

For your information, disabling the Hamachi network adapter was not enough, because it seems that its Windows service is re-enabling the adapter.

Also, reconnecting to my Hamachi network did not solve the problem. Only disabling the adapter (by disabling the Windows LogMeIn Hamachi service) or possibly removing Hamachi solved the problem for me.

Can I ask HttpWebRequest to exit through a specific network adapter?

Answer 11

We had the same problem in a web application. We waited for an answer for 5 seconds. When we change the user to applicationPool on IIS on networkService, the response began to arrive less than 1 second