Geek Answers Handbook

Does System.Net.Http.HttpClient suffer from HttpWebRequest.AllowWriteStreamBuffering?

I'm trying to use System.Net.Http.HttpClient to POST a larger file (+ 1 GB), but it throws a SystemOutOfMemory exception:

 at System.Net.ScatterGatherBuffers.AllocateMemoryChunk(Int32 newSize) at System.Net.ScatterGatherBuffers..ctor(Int64 totalSize) at System.Net.ConnectStream.EnableWriteBuffering() at System.Net.HttpWebRequest.SetRequestSubmitDone(ConnectStream submitStream) at System.Net.Connection.CompleteStartRequest(Boolean onSubmitThread, HttpWebRequest request, TriState needReConnect) at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit) at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName) at System.Net.HttpWebRequest.SubmitRequest(ServicePoint servicePoint) at System.Net.HttpWebRequest.BeginGetRequestStream(AsyncCallback callback, Object state) at System.Net.Http.HttpClientHandler.StartGettingRequestStream(RequestState state) at System.Net.Http.HttpClientHandler.PrepareAndStartContentUpload(RequestState state)

Apparently, a similar problem occurs with HttpWebRequest , as discussed here: http://support.microsoft.com/kb/908573 .

Is it possible to set AllowWriteStreamBuffering basic web request to false ? I can not find.

Greetings

+4
source share
4 answers

Just to save time for the interests of others, I answer my question.

After several tests, the exception seems to be the same issue with HttpWebRequest , as discussed in the question. I am using Microsoft.AspNet.WebApi version 4.0.20710.0.

The following are two equivalent code snippets; the former does not work on large files, while the latter works fine.

By the way, despite the fact that the general benefits of HttpClient are becoming apparent HttpClient

using HttpClient

 var clientRef = new System.Net.Http.HttpClient( new HttpClientHandler() { Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword) }); clientRef.BaseAddress = new Uri(serverAddress); clientRef.DefaultRequestHeaders.ExpectContinue = false; clientRef.PostAsync( MyFavoriteURL, new System.Net.Http.StreamContent(inputStream)).ContinueWith( requestTask => { HttpResponseMessage response = requestTask.Result; response.EnsureSuccessStatusCode(); }, TaskContinuationOptions.LongRunning).Wait();

using HttpWebRequest

 // Preauthenticate var req = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL); req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword); req.Method = "POST"; req.PreAuthenticate = true; req.Timeout = 10000; using (var resp = (System.Net.HttpWebResponse)req.GetResponse()) { if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK) { throw new Exception("Authentication error"); } } // Upload req = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL); req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword); req.Method = "POST"; req.PreAuthenticate = true; req.Timeout = 1200000; req.ContentLength = inputStream.Length; req.ContentType = "application/binary"; req.AllowWriteStreamBuffering = false; req.Headers.ExpectContinue = false; using (var reqStream = req.GetRequestStream()) { inputStream.CopyTo(reqStream); } using (var resp = (System.Net.HttpWebResponse)req.GetResponse()) { if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK) { throw new Exception("Error uploading document"); } }
+3
source

I also ran into the same problem for large file uploads. Adding my results so that they help someone.

HttpClient determines whether to do buffering based on the following conditions:

 if( HttpRequestMessage.Headers.TransferEncodingChunked == true || HttpRequestMessage.Content.Headers.ContentLength != null ) { //do streamed upload }else { //do buffered upload. }

I used PushedStreamContent as HttpRequestMessage.Content because my server accepts a maximum of 150 MB per request. As soon as I set TransferEncodingChunked to True, the memory drop will be reduced.

0
source

I think you have a duplicate header, you can delete the previous one

 req.Headers.Add("ExpectContinue", "false"); ... req.Headers.ExpectContinue = false;
0
source

Helped me send a large file without buffering through the "relay controller".
The scenario that I had to work:
Web-based client application (spa, etc.) -> RelayController -> The ultimate content controller that stores the http data stream

  var request = new HttpRequestMessage() { RequestUri = new Uri("https://someurl/api/upload"), Method = HttpMethod.Post, }; request.Headers.TransferEncodingChunked = true; request.Content = new StreamContent(Request.Body); await httpClient.SendAsync(request);`

Spent a lot of time getting it to work, got the lead from Prakash P, who was streaming httpclient vs buffering.

0
source

More articles:


All Articles