How to limit response length using http.request in Node.js

Question

So, in this (simplified) bit of code, when someone clicks on my node server, I make a GET request to another site and print the header of the HTML page to the console. Works great:

var http = require("http"); var cheerio = require('cheerio'); var port = 8081; s = http.createServer(function (req, res) { var opts = { method: 'GET', port: 80, hostname: "pwoing.com", path: "/" }; http.request(opts, function(response) { console.log("Content-length: ", response.headers['content-length']); var str = ''; response.on('data', function (chunk) { str += chunk; }); response.on('end', function() { dom = cheerio.load(str); var title = dom('title'); console.log("PAGE TITLE: ",title.html()); }); }).end(); res.end("Done."); }).listen(port, '127.0.0.1'); 

However, in a real application, users can specify the URL to be clicked. This means that my node server can upload 20 GB movie files or something else. Not good. The content header is not intended to stop this, as it is not transmitted by all servers. Then the question is:

How can I say to stop a GET request, say the first 10 KB were received?

Hooray!

Answer 1

You can abort the request as soon as you read enough data:

  http.request(opts, function(response) { var request = this; console.log("Content-length: ", response.headers['content-length']); var str = ''; response.on('data', function (chunk) { str += chunk; if (str.length > 10000) { request.abort(); } }); response.on('end', function() { console.log('done', str.length); ... }); }).end(); 

This will lead to an interruption of the request of approximately 10,000 bytes, as the data arrives in pieces of different sizes.