WebRequest / WebResponse memory leak

Question

I have a .Net Framework # 4.0 application that makes a large number of web requests using the WebRequest / WebResponse classes, since I see that it has a memory leak (or maybe I'm doing something wrong) I wrote a little A small application that demonstrates this:

class Program { public static void Main(string[] args) { while(true) { var webRequest = (HttpWebRequest)WebRequest.Create("http://www.gooogle.com"); Init(webRequest); using (var webResponse = (HttpWebResponse)webRequest.GetResponse()) { var responseStream = webResponse.GetResponseStream(); responseStream.ReadTimeout = 30; var streamReader = new StreamReader(responseStream, Encoding.UTF8); var page = streamReader.ReadToEnd(); streamReader.Close(); streamReader.Dispose(); responseStream.Close(); responseStream.Dispose(); webResponse.Close(); Console.WriteLine("Done"); //GC.Collect(); } } } private static void Init (HttpWebRequest webRequest) { webRequest.Method = "GET"; webRequest.Host = "www.gooogle.com"; webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB6.5; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; .NET4.0C; .NET4.0E; InfoPath.3) chromeframe/5.0.375.62"; webRequest.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*"; webRequest.KeepAlive = true; } } 

The only solution I came across is using GC.Collect () (not noted in the example). All objects are located, all threads are closed, am I missing something?

I found something, but I don’t understand the reason, if I minimize the console, the memory usage will decrease and it looks OK, which may be the reason that there is a problem with Conosole or WinForm, how can I fix it?

Answer 1

You allocate memory in a narrow cycle. You probably don't have a memory leak, you have a bad application (in the sense that it uses a lot of system resources for no reason.)

The garbage collector is not going to interrupt your loop to compile free memory if it is not related to memory pressure. A simple fix would be to introduce a delay between iterations of the loop (it might be as simple as Thread.Sleep, although I do not recommend doing this.)

As soon as your program does not work so hard to consume all available processor time, it should allow the GC to work more often.

Answer 2

Try the following:

 while (true) { var webRequest = (HttpWebRequest) WebRequest.Create("http://www.gooogle.com"); Init(webRequest); using (var webResponse = (HttpWebResponse) webRequest.GetResponse()) { using (var responseStream = webResponse.GetResponseStream()) { responseStream.ReadTimeout = 30; using (var streamReader = new StreamReader(responseStream, Encoding.UTF8)) { var page = streamReader.ReadToEnd(); } } Console.WriteLine("Done"); } } 

Answer 3

Yes, we found a leak. Solution ... be careful what you do with this thing. Think do not use it. I believe he has ... problems (at least in 3.5, presumably other versions too). Oh, and don't forget to report it / vote for it on Microsoft Connect.

Answer 4

Look at this ( source ):

symptoms

When you use the HttpWebRequest class to send large amounts of data for an HTTP POST or PUT request, the request may fail on a computer running Microsoft.NET Framework. In addition, you may get a low memory exception.

You may notice that an application using the HttpWebRequest class consumes a lot of memory. When using the system monitor to monitor an application using the HttpWebRequest class, the number of private bytes will continue to increase as data is sent.

cause

This problem occurs because the .NET Framework by default buffers outgoing data when using the HttpWebRequest class. KB article http://support.microsoft.com/kb/908573 describes the original problem.

resolution

To work around this problem, set the HttpWebRequest.AllowWriteStreamBuffering property to false . In this case, outgoing data (object body) for a POST or PUT request will not be buffered in memory.

In versions of the Microsoft .NET Framework earlier than 4.5, setting the HttpWebRequest.AllowWriteStreamBuffering property of the HttpWebRequest.AllowWriteStreamBuffering false property can sometimes lead to errors when loading data to verified endpoints. For example, you may encounter a System.Net.WebException with the message "Data buffering is required for this request." However, in a deeper investigation, the response related to the exception actually indicates the status code System.Net.HttpStatusCode.Unauthorized (401). The knowledge base article http://support.microsoft.com/kb/908573 describes a workaround for preauthentication and KeepAlive connections to handle 401 response.

Unlike Microsoft.NET Framework 1.1, 2.0, 3.0, 3.5, and 4.0, Microsoft.NET Framework 4.5 adds new design functionality to the HttpWebRequest.AllowWriteStreamBuffering property. The new function can process the authentication script directly if the Expect100Continue function is enabled. The default value of ServicePointManager.Expect100Continue is true .