Read file from server with some offset

Question

How can I read a file from the server starting at some offset (similar behavior to wget -c )? What headers should I send to the server? What futures should the server support?

Answer 1

The request should use the Range header. But you can only use it if the server tells you that it is accepting a range request from the Accept-Ranges response header.

This is an example session. Suppose we are interested in getting a portion of this image . First, we send an HTTP HEAD request to determine: a) if the server supports byte ranges, b) content length:

 > HEAD /2238/2758537173_670161cac7_b.jpg HTTP/1.1 > Host: farm3.static.flickr.com > Accept: */* > < HTTP/1.1 200 OK < Date: Thu, 08 Jul 2010 12:22:12 GMT < Content-Type: image/jpeg < Connection: keep-alive < Server: Apache/2.0.52 (Red Hat) < Expires: Mon, 28 Jul 2014 23:30:00 GMT < Last-Modified: Wed, 13 Aug 2008 06:13:54 GMT < Accept-Ranges: bytes < Content-Length: 350015 

Then we send a GET request with a Range header requesting the first 11 bytes of picure:

 > GET /2238/2758537173_670161cac7_b.jpg HTTP/1.1 > Host: farm3.static.flickr.com > Accept: */* > Range: bytes=0-10 > < HTTP/1.1 206 Partial Content < Date: Thu, 08 Jul 2010 12:26:54 GMT < Content-Type: image/jpeg < Connection: keep-alive < Server: Apache/2.0.52 (Red Hat) < Expires: Mon, 28 Jul 2014 23:30:00 GMT < Last-Modified: Wed, 13 Aug 2008 06:13:54 GMT < Accept-Ranges: bytes < Content-Range: bytes 0-10/350015 < Content-Length: 11 < 

This is the hexadecimal dump of the first 11 bytes:

 00000000 ff d8 ff e0 00 10 4a 46 49 46 00 |......JFIF.| 0000000b 

See the Range Header Specification in HTTP RFC 2616 for details.

Answer 2

At http://www.gnu.org/software/wget/manual/wget.html

Note that '-c only works with ftp servers and with http servers that support the Range header.

At http://tools.ietf.org/html/rfc2616

Examples of the byte range of the value specifier (provided that the body object is 10000 in length):

  - The first 500 bytes (byte offsets 0-499, inclusive): bytes=0- 499 - The second 500 bytes (byte offsets 500-999, inclusive): bytes=500-999 - The final 500 bytes (byte offsets 9500-9999, inclusive): bytes=-500 - Or bytes=9500- - The first and last bytes only (bytes 0 and 9999): bytes=0-0,-1 - Several legal but not canonical specifications of the second 

500 bytes (byte offsets 500-999 inclusive): bytes = 500-600,601-999 bytes = 500-700,601-999

So you have to send

 Range:bytes=9500- 

To check if the server supports it, you can test the acceptance range as such

Origin servers accepting byte requests can send

Accept-Ranges: bytes

but this is not required. Clients MAY generate byte range requests without getting this header for the resource involved. Range units are defined in section 3.12.

Servers that do not accept any range request for a resource MAY send

 Accept-Ranges: none 

to tell the client not to request a range request.