HEAD and GET request

Question

I always had the idea that the HEAD request instead of the GET request was faster (regardless of the size of the resource) and, therefore, had advantages in some solutions.

However, when creating a HEAD request in Python (for a dynamically generated 5+ MB resource), I realized that it took the same time as the GET request (I was hoping for almost 27 seconds instead of “less than 2” seconds).

Used some urllib2 solutions for the HEAD request found here, and even used pycurl (setting headers and nobody is True ). Both of them took the same time.

Am I losing something conceptually? Is it possible using Python to execute a "fast" HEAD request?

Answer 1

The server takes up most of the time, not your requester or network. If this is a dynamic resource, the server probably does not know all the header information, in particular the Content-Length, until it builds it. Thus, all this is needed to create everything, regardless of whether you are performing HEAD or GET.

Answer 2

The response time depends on the server, not on your request. A HEAD request returns less data (only headers), so conceptually it should be faster, but in practice many static resources are cached, so there is practically no measurable difference (just the time when additional packets go off the wire).

Answer 3

Most likely, the bulk of this request time is actually any process that generates a 5 + MB response on the server, rather than the time it was sent.

In many cases, the web application will still execute the full script when responding to the HEAD request - it simply won’t send the full body back to the requestor.

If you have access to the code that processes this request, you can add a condition there to make it process the request differently depending on the method, which can speed it up significantly.