HttpWebRequest and HttpWebResponse Progress Tracking

Question

I use the Sharefile API, which sends HTTP requests and receives corresponding responses. They are created at the URL and always use the same function. Here it is.

Private Function InvokeShareFileOperation(ByVal requestUrl As String) As JObject Dim request As HttpWebRequest = WebRequest.Create(requestUrl) Dim response As HttpWebResponse = request.GetResponse() Dim reader As StreamReader = New StreamReader(response.GetResponseStream()) Dim json As String = reader.ReadToEnd() response.Close() Return JObject.Parse(json) End Function 

Since some operations are a bit long, I need to somehow track their progress while they go, and do not know how to do it. Later, I intend to use this progress and create a progress bar.

(EDIT) By the way, this is the second line of code (below), which takes more time, that is, the operation to track.

 Dim response As HttpWebResponse = request.GetResponse() 

Answer 1

EDIT: I don’t think you can measure the progress in any exact way here, since the main part of the operation seems to depend on the remote server processing the request. GetResponse () handles the DNS setup, connection, sending and waiting for a remote server, and that's all out of your hands. Reading the response flow is only measurable if the content length header is returned. Personally, I would show progress as 20% initially, 60% when GetResponse returns, and then the last 40% can be displayed gradually if you have the length of the content before loading or done in one pass, as soon as you finish reading the response stream.

As its web request, you can first find out the length of the content and then read the response stream using a buffer instead of ReadToEnd (). This allows you to calculate progress and turn off notifications when downloading a response.

 Dim request As HttpWebRequest = WebRequest.Create(requestUrl) Using response As HttpWebResponse = request.GetResponse() Dim contentLength As Long = response.ContentLength Dim bytesReceived As Long Dim bufferLength as Integer = 8192 Dim buffer(bufferLength) As Char Dim sb As New StringBuilder Using reader As StreamReader = New StreamReader(response.GetResponseStream()) Do Dim bufferedCount As Integer = reader.Read(buffer, 0, bufferLength) sb.Append(buffer, 0, bufferedCount) bytesReceived += bufferedCount Console.WriteLine(bytesReceived / contentLength * 100 & "%") Loop While bytesReceived < contentLength End Using Return JObject.Parse(sb.ToString) End Using 

Obviously, you can replace Console.WriteLine with the progress update function or by calling the SignalR hub to refresh the web page, and you can experiment with the buffer size to find out what works best for you.

Answer 2

First we need to find out what is slowing down. The request is not sent until GetResponse() is GetResponse() , so server processing may take some time. Download may take some time. If the answer is small (regarding connection speed), you cannot do much (you can if the server belongs to you, but we will focus on the client) because you cannot make progress from the server. If the answer is large and you want to track the download, you can only do this if you have a Content-Length header. To receive only headers, the server must support the HEAD request method. So here is the code:

 Imports System Imports System.Net Imports System.IO Imports System.Text Imports System.Threading Imports Microsoft.VisualBasic Public Class Form1 Private Function InvokeShareFileOperation(ByVal requestUrl As String) As JObject HTTPWebRequest_GetResponse.Main(requestUrl) ProgressBar1.Value = 0 Dim result As String Do Try ProgressBar1.Value = HTTPWebRequest_GetResponse.progress Catch ex As ArgumentOutOfRangeException ProgressBar1.Style = ProgressBarStyle.Marquee End Try If HTTPWebRequest_GetResponse.done = True Then result = HTTPWebRequest_GetResponse.response ProgressBar1.Style = ProgressBarStyle.Continuous ProgressBar1.Value=100 Debug.WriteLine(result) Return JObject.Parse(result) Exit Do End If Loop End Function End Class Public Class RequestState ' This class stores the State of the request. Private BUFFER_SIZE As Integer = 1024 Public requestData As StringBuilder Public BufferRead() As Byte Public request As HttpWebRequest Public response As HttpWebResponse Public streamResponse As Stream Public Sub New() BufferRead = New Byte(BUFFER_SIZE) {} requestData = New StringBuilder("") request = Nothing streamResponse = Nothing End Sub 'New End Class 'RequestState Class HTTPWebRequest_GetResponse Private BUFFER_SIZE As Integer = 1024 Public Shared response As String Public Shared done As Boolean = False Public Shared length As Long = 1 Public Shared progress As Integer Public Shared myHttpWebRequest As HttpWebRequest Public Shared myRequestState As New RequestState() Shared Sub Main(url As String) Try Dim headRequest As HttpWebRequest = WebRequest.Create(url) headRequest.Method = "HEAD" Dim headResponse As HttpWebResponse = headRequest.GetResponse length = headResponse.ContentLength Debug.WriteLine(length) headResponse.Close() ' Create a HttpWebrequest object to the desired URL. myHttpWebRequest = WebRequest.Create(url) ' Create an instance of the RequestState and assign the previous myHttpWebRequest ' object to its request field. myRequestState.request = myHttpWebRequest 'Dim myResponse As New HTTPWebRequest_GetResponse() ' Start the asynchronous request. Dim result As IAsyncResult = CType(myHttpWebRequest.BeginGetResponse(New AsyncCallback(AddressOf RespCallback), myRequestState), IAsyncResult) Catch e As WebException Debug.WriteLine("Main Exception raised!") Debug.WriteLine("Message: " + e.Message) Debug.WriteLine("Status: " + e.Status) Catch e As Exception Debug.WriteLine("Main Exception raised!") Debug.WriteLine("Source : " + e.Source) Debug.WriteLine("Message : " + e.Message) End Try End Sub 'Main Private Shared Sub RespCallback(asynchronousResult As IAsyncResult) Debug.WriteLine("RespCallBack entered") Try ' State of request is asynchronous. Dim myRequestState As RequestState = CType(asynchronousResult.AsyncState, RequestState) Dim myHttpWebRequest As HttpWebRequest = myRequestState.request myRequestState.response = CType(myHttpWebRequest.EndGetResponse(asynchronousResult), HttpWebResponse) ' Read the response into a Stream object. Dim responseStream As Stream = myRequestState.response.GetResponseStream() myRequestState.streamResponse = responseStream ' Begin the Reading of the contents of the HTML page. Dim asynchronousInputRead As IAsyncResult = responseStream.BeginRead(myRequestState.BufferRead, 0, 1024, New AsyncCallback(AddressOf ReadCallBack), myRequestState) Return Catch e As WebException Debug.WriteLine("RespCallback Exception raised!") Debug.WriteLine("Message: " + e.Message) Debug.WriteLine("Status: " + e.Status) Catch e As Exception Debug.WriteLine("RespCallback Exception raised!") Debug.WriteLine("Source : " + e.Source) Debug.WriteLine("Message : " + e.Message) End Try End Sub 'RespCallback Private Shared Sub ReadCallBack(asyncResult As IAsyncResult) Debug.WriteLine("ReadCallBack entered") Try Dim myRequestState As RequestState = CType(asyncResult.AsyncState, RequestState) Dim responseStream As Stream = myRequestState.streamResponse Dim read As Integer = responseStream.EndRead(asyncResult) ' Read the HTML page. If read > 0 Then myRequestState.requestData.Append(Encoding.ASCII.GetString(myRequestState.BufferRead, 0, read)) If length = -1 Or length = 0 Then progress = -1 Else progress = myRequestState.BufferRead.Length * 100 / length Debug.WriteLine(progress) End If Dim asynchronousResult As IAsyncResult = responseStream.BeginRead(myRequestState.BufferRead, 0, 1024, New AsyncCallback(AddressOf ReadCallBack), myRequestState) Else If myRequestState.BufferRead.Length > 1 Then Dim fullResponse As String = myRequestState.requestData.ToString response = fullResponse.Substring(0, fullResponse.IndexOf("</body>")).Substring(fullResponse.IndexOf(">", fullResponse.IndexOf("<body")) + 2) 'Returns only body ' Release the HttpWebResponse resource. myRequestState.response.Close() done = True Debug.WriteLine(done) End If responseStream.Close() End If Catch e As WebException Debug.WriteLine("ReadCallBack Exception raised!") Debug.WriteLine("Message: " + e.Message) Debug.WriteLine("Status: " + e.Status) Catch e As Exception Debug.WriteLine("ReadCallBack Exception raised!") Debug.WriteLine("Source : " + e.Source) Debug.WriteLine("Message : " + e.Message) End Try End Sub 'ReadCallBack End Class 'HttpWebRequest_BeginGetResponse 

I took the code from http://msdn.microsoft.com/en-us/library/debx8sh9(v=vs.110).aspx and changed it.

EDIT: Code now returns only the body and the response is closed.

EDIT2: As @ Geezer68 said, it is not 100% more accurate, but it is ok, showing the progress to the user.

Answer 3

I'm sure you need reader.BaseStream.Length so that you can find out the length before reading. (At least I did, so I tried) But he threw a NotSupportedException with the message This stream does not support seek operations . So, I googled StreamReader + This stream... and found this SO link:

Error "This thread does not support search operations" in C #

So the short answer is: This is not possible.

Answer 4

Maybe a simple stopwatch is a way to start with?

  Dim timer As System.Diagnostics.Stopwatch = New Stopwatch() Dim request As HttpWebRequest = WebRequest.Create(requestUrl) timer.Start() Dim response As HttpWebResponse = request.GetResponse() timer.Stop() Dim reader As StreamReader = New StreamReader(response.GetResponseStream()) Dim json As String = reader.ReadToEnd() response.Close() Label1.Text = "Secs:" & timer.Elapsed.ToString() 

Answer 5

Here is a link to a Microsoft example where you set the buffer size and callback of the response object https://msdn.microsoft.com/en-us/library/86wf6409%28v=vs.110%29.aspx