Geek Answers Handbook

Fastest way to load multiple web pages using C #

This is a (basic) example of what I have:

foreach (var uri in uris) { using (var client = new WebClient()) { client.Proxy = null; client.DownloadStringCompleted += DownloadComplete; client.DownloadStringAsync(uri); } }

Is there a faster way?

+4
source share
2 answers

It is important to do the parallel downloads that you are already doing thanks to the Async download.

The download speed of your code depends entirely on the actual speed of transmission over the network, so it is as good as it is.

+9
source

I believe that you can do this much faster if you set the Accept-Encoding header for gzip, deflate, if the server supports gzip (a modern web server should support).

The basic idea is to ask the server to pin content before downloading, usually for a shared web page, you can get 50% smaller in size and therefore you can save 50% of the time.

Take a look at this: http://csharpfeeds.com/post/5518/HttpWebRequest_and_GZip_Http_Responses.aspx

+2
source

More articles:


All Articles