Tomcat gzip while problem

Question

I am running some problems with one of my data source services. As the HTTP response headers say, it runs on Apache-Coyote / 1.1. The server gives answers with Transfer-Encoding: chunked, here is an example response:

HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Content-Type: text/xml;charset=utf-8 Transfer-Encoding: chunked Content-Encoding: gzip Date: Tue, 30 Mar 2010 06:13:52 GMT 

And the problem is that when I ask the server to send a gzip request, it often sends an incomplete response. I get the answer, I see that the last piece is received, but after unrolling, I see that the answer is partial. I have never seen this behavior with gzip disabled in request headers.

So my question is: is this a common tomcat problem? maybe one of them that does compression? Or maybe it could be some kind of proxy problem? I can’t talk about tomcat versions or what type of gzip they use, but feel free to ask, I will try to ask my service provider.

Thanks.

Answer 1

Since the length of the contents of the gzipped response is unpredictable and potentially costly and slow to compress it completely in memory first, then calculate the length and then transfer the gzipped response from memory, the average web server will send them to pieces using Transfer-Encoding: chunked without Content-Length headline.

Since this is a home HTTP client, it sounds as if it is not handling ordered requests correctly. You must define the header of the Transfer-Encoding response, and if it is chunked , then you need to parse it as a fragmented stream.

You can learn from the above links of the HTTP specification and from Wikipedia how to parse a stream with channels. Each fragment consists of a header indicating the length of the fragment in hexadecimal, then CRLF, and then the actual contents of the fragment, and then CRLF. This is repeated until the fragment with the heading indicates the length of block 0 . You need to disassemble the pieces separately and then glue them together.

To save all the tedious coding work (probably also for the rest of your homegrown HTTP client), I highly recommend looking at the Apache HttpComponents Client .