How to read response flow before completing Http response

Question

When executing a request using the HttpWebRequest object, I need to call the GetResponse () method to send the request and get a response.
The problem with this method is that it does not return a response object until all data has been received. Let's say I upload a 100 MB file, I can’t read it until the response is complete and only 100 MB.
I want them to be able to read the bytes of the response stream as soon as they arrive without waiting for the response to complete.
I know that I can use the Range Http header, but it will not work in my situation.

Answer 1

I think this is very close to what @ Zachary offers. And this (seems) work (s); in fact, I think using using , as @Zachary does even “better.”
The main thing is that I cannot see the GetResponse() lock behavior that you (seem to be) describing.

In addition, the following code only roughly shows how everything works; he will not read the stream to the end, for example (if not a coincidence :)). But it should work if you copy-n-paste it into an empty Console Application project in Visual Studio.

You can try using some “shorter” URL for the test. In this example, the ISO of the debian distribution is downloaded (bit over 600 MB). Sorry, debian, I didn’t want to steal your bandwidth. → Btw: is there anything reasonable that can be used to test such a scenario?

The code is strongly inspired by C # - how to read a continuous stream of XML over HTTP .

 namespace StreamReadWebRequest { using System; using System.Collections.Generic; using System.Text; using System.Net; using System.IO; class Program { static void Main(string[] args) { HttpWebRequest req; HttpWebResponse res = null; try { req = (HttpWebRequest)WebRequest.Create( "http://cdimage.debian.org/debian-cd/5.0.4/i386/iso-cd/debian-504-i386-CD-1.iso"); res = (HttpWebResponse)req.GetResponse(); Stream stream = res.GetResponseStream(); byte[] data = new byte[4096]; int read; while ((read = stream.Read(data, 0, data.Length)) > 0) { Process(data, read); } } finally { if (res != null) res.Close(); } Console.In.Read(); } private static void Process(byte[] data, int read) { Console.Out.Write(ASCIIEncoding.ASCII.GetString(data)); } } } 

Answer 2

I was looking for the same thing: server threads shared XML data, and I needed a C # client that could access that data during server streaming. I tried many different ways to access the source (WebChannelFactory, WebClient, HttpWebRequest / Response, TcpClient), but still failed. Finding this topic, I focused on HttpWebRequest / Response, where I have the same problem that the following line blocks:

 HttpWebResponse resp = (HttpWebResponse)request.GetResponse(); 

As Artion stated to Chilaru, if he blocks: something is not right, because it should not. Now, focusing on trying to replicate the default behavior when loading large .ISO files, I found that Fiddler blocked the GetResponse () method!

However, there is no problem opening Fiddler after the thread has been configured (i.e. GetResponse () has already been called), but during HTTP GET, if you find that GetResponse () blocks the attempt to close Fiddler and see the application now continuously executes a normal stream (i.e. reads a stream).

Answer 3

If you set the size of the buffer when reading, you can read the data in pieces ... example ...

  // Get the response stream using(Stream resStream = response.GetResponseStream()) { string parseString = null; int count = 0; do { // Read a chunk of data count = resStream.Read(buf, 0, buf.Length); if (count != 0) { // Convert to ASCII parseString = Encoding.ASCII.GetString(buf, 0, count); // Append string to results sb.Append(tempString); } } while (count > 0); } 

Answer 4

I'm not sure what you have on your side, but I know that it is a fact (and I'm sure many people will agree here) that GetResponse() will NOT load the whole file back. He will send a request, wait for a response and receive the response headers.

After the response, you can easily get the response stream with GetResponseStream() , which is the actual data stream that is downloaded from the server. And you can easily access the response stream before the entire file is uploaded. This is 100% true and verified.

If you don’t get the same behavior (which is really strange and should not happen), can you add some code that doesn’t work, as I explained above?

Also check out the example posted by scherand . This proves once again that it works perfectly, without any special hacks.