Transmission Encoding: gzip and Content-Encoding: gzip

Question

What is the current state of affairs when it comes to

Transfer-Encoding: gzip 

or

 Content-Encoding: gzip 

when I want to allow clients , for example. a limited bandwidth until signals ready to receive a compressed response , and the server has a final opinion on whether to compress .

The latter is what, for example, Apache mod_deflate and IIS do if you let it take care of the compression. Depending on the size of the compressed content, it will do additional Transfer-Encoding: chunked .

It will also include Vary: Accept-Encoding , which is already hinting at a problem. Content-Encoding seems to be part of the entity, so changing the Content-Encoding is reduced to changing the object, that is, another Accept-Encoding header, for example, the cache cannot use its cached version of an identical identical object.

Is there a definite answer to this question that I missed (and which did not bury itself inside the message in a long stream in some kind of apache newsgroup)?

My real impression:

• Transfer-coding will actually be the right way to do what is mainly done with Content-Encoding, using existing server and client implants.
• Content-Encoding has several problems due to its semantic consequences (what should the server do with ETag when transparently compressing the response?)
• Reason chicken'n'egg: browsers do not support it because servers are not because browsers are not

So, I assume that the correct path will be Transfer-Encoding: gzip (or, if I additionally become a piece of the body, it will become Transfer-Encoding: gzip, chunked ). And there is no reason to touch Vary or ETag or any other heading in this case, since this is a transport layer thing.

At the moment, I'm not too worried about the "hop-by-hop" from Transfer-Encoding , something that others seem to bother with in the first place, because proxies can unpack and forward uncompressed to the client, However, proxies can also redirect it as-is (compressed) if the original request has the correct Accept-Encoding header, which in the case of all browsers that I know are data.

Btw, this problem is at least a decade, see for example https://bugzilla.mozilla.org/show_bug.cgi?id=68517 .

Any clarification on this will be appreciated. Both in terms of what is considered standard, and in terms of practicality. For example, HTTP client libraries that support only transparent "Content-Encoding" will be an argument against practicality.

Answer 1

Quote Roy T. Fielding , one of the authors of RFC 2616:

changing the encoding of content "on the fly" inconsistently (neither "never" nor "always") makes subsequent requests regarding this content (for example, PUT or conditional GET) impossible to process correctly. This, of course, is why doing on-the-fly content encoding is a dumb idea, and why I added Transfer-Encoding for HTTP as the right way to do on-the-fly encoding without changing the resource.

Source: https://issues.apache.org/bugzilla/show_bug.cgi?id=39727#c31

In other words: don't run Content-Encoding on the fly , use Transfer-Encoding instead!

Edit: That is , if you do not want to serve gzipped content for clients who understand Content-Encoding . Unfortunately, most of them. But keep in mind that you are leaving the scope of the specification and may encounter problems such as those mentioned by Fielding, as well as others, for example. when caching proxies are involved.

Answer 2

Using the correct one , as defined in RFC 2616 and actually implemented in the wild, is that the client sends an Accept-Encoding request header (the client can specify multiple encodings). Then the server can and only then encode the response in accordance with the encodings supported by the client (if the file data has not yet been saved in this encoding), indicate in the response header Content-Encoding which encoding is used. The client can then read the socket data based on Transfer-Encoding (i.e. chunked ), and then decode it based on Content-Encoding (i.e.: gzip ).

So, in your case, the client will send the Accept-Encoding: gzip request header, and then the server may decide to compress (if not already) and send the Content-Encoding: gzip response header and optionally Transfer-Encoding: chunked .

And yes, the Transfer-Encoding header can be used in requests, but only for HTTP 1.1, which requires both client and server implementations to support chunked encoding in both directions.

ETag uniquely identifies resource data on a server, not the data that is actually transmitted. If this URL resource changes the ETag value, it means that the server-side data for this resource has changed.