Create a zip file on the fly for download via node.js

Question

I just need to do the setup below using a node js script (generate a zip "on the fly" without touching the disk and respond to client requests for download). Can anyone be guided and send your work scripts. I tried searching on the internet, it seems we can achieve this through zipstream. But did not find any script example / work.

• capture files matching * .xml files from the root folder.

• Immediately writes http-response http-headers to the client to tell it to download, and the file name is .zip.

• zipstream writes the bytes of the header of the zip container.

• Creates an HTTP request to the first image in S3.

• Pipes that are in zipstream (we do not need to run deflate, since the images are already compressed).

• Pipes that are part of an HTTP client request.

• Repeats for each image, with zipstream correctly writing envelopes for each file.

• zipstream writes footer bytes for zip container

• Completes the http response.

Thanks,

Shrinivas

Answer 1

I had the same requirement ... stream files from Amazon S3, zip them on the fly (in memory) and deliver them to the browser via node.js. My solution included using knox and archiver packages and log archive bytes into the result stream.

Since this is on the fly, you won’t know the archive size you received, and therefore you won’t be able to use the “Content-Length” HTTP header. Instead, you will have to use the heading "Transfer-Encoding: chunked".

The disadvantage of "chunked" is that you will not get a progress bar to download. I tried setting the Content-Length header to an approximate value, but this only works for Chrome and Firefox; IE corrupts the file; not tested with Safari.

var http = require("http"); var knox = require("knox"); var archiver = require('archiver'); http.createServer(options, function(req, res) { var zippedFilename = 'test.zip'; var archive = archiver('zip'); var header = { "Content-Type": "application/x-zip", "Pragma": "public", "Expires": "0", "Cache-Control": "private, must-revalidate, post-check=0, pre-check=0", "Content-disposition": 'attachment; filename="' + zippedFilename + '"', "Transfer-Encoding": "chunked", "Content-Transfer-Encoding": "binary" }; res.writeHead(200, header); archive.store = true; // don't compress the archive archive.pipe(res); client.list({ prefix: 'myfiles' }, function(err, data) { if (data.Contents) { var fileCounter = 0; data.Contents.forEach(function(element) { var fileName = element.Key; fileCounter++; client.get(element.Key).on('response', function(awsData) { archive.append(awsData, {name: fileName}); awsData.on('end', function () { fileCounter--; if (fileCounter < 1) { archive.finalize(); } }); }).end(); }); archive.on('error', function (err) { throw err; }); archive.on('finish', function (err) { return res.end(); }); } }).end(); }).listen(80, '127.0.0.1');