Proper streaming and input and output in HTTP Servlet

Question

I am trying to write a servlet that will handle a POST request and pass both input and output data. I mean, he should read one line of input, do some work on that line, and write one line of output. And it should be able to handle an arbitrary long request (so it will also produce an arbitrary long response) without exceptions from memory. Here is my first attempt:

protected void doPost(HttpServletRequest request, HttpServletResponse response) {
    ServletInputStream input = request.getInputStream();
    ServletOutputStream output = response.getOutputStream();

    LineIterator lineIt = lineIterator(input, "UTF-8");
    while (lineIt.hasNext()) {
        String line = lineIt.next();
        output.println(line.length());
    }
    output.flush();
}

Now I checked this servlet with curland it works, but when I wrote a client using Apache HttpClient, the client thread and server hang. The client looks like this:

HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(...);

// request
post.setEntity(new FileEntity(new File("some-huge-file.txt")));
HttpResponse response = client.execute(post);

// response
copyInputStreamToFile(response.getEntity().getContent(), new File("results.txt"));

. - . , ( - ), , . , , , .

, curl , - ( ?). , , Apache HttpClient , curl?

: , ? - , , , :

ServletInputStream input = request.getInputStream();
ServletOutputStream output = response.getOutputStream();

// prepare intermediate store
int threshold = 100 * 1024; // 100 kB before switching to file store
File file = File.createTempFile("intermediate", "");
DeferredFileOutputStream intermediate = new DeferredFileOutputStream(threshold, file);

// process request to intermediate store
PrintStream intermediateFront = new PrintStream(new BufferedOutputStream(intermediate));
LineIterator lineIt = lineIterator(input, "UTF-8");
while (lineIt.hasNext()) {
    String line = lineIt.next();
    intermediateFront.println(line.length());
}
intermediateFront.close();

// request fully processed, so now it time to send response
intermediate.writeTo(output);

file.delete();

, , , , , curl, . , , .

, / , :

• , ( DeferredFileOutputStream).

, , .

, ? , , , - ...

Answer 1

Jetty HttpClient http://www.eclipse.org/jetty/documentation/current/http-client-api.html

.

HttpClient httpClient = new HttpClient();
httpClient.start();

Request request = httpClient.newRequest("http://localhost:8080/line-lengths");
final OutputStreamContentProvider contentProvider = new OutputStreamContentProvider();
InputStreamResponseListener responseListener = new InputStreamResponseListener();

request.content(contentProvider).method(HttpMethod.POST).send(responseListener); //async request
httpClient.getExecutor().execute(new Runnable() {
    public void run() {
        try (OutputStream outputStream = contentProvider.getOutputStream()) {
            writeRequestBodyTo(outputStream); //writing to stream in another thread
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
});

readResponseBodyFrom(responseListener.getInputStream()); //reading response
httpClient.stop();