Transaction Using NodeJS Multer

Question

I am currently implementing a file and image download service for my users. I want to convert these images (resize / optimize) before uploading to my s3 bucket.

What I'm doing now: using a multi-part form on my interface (I think the actual implementation does not matter here.) And the multer and multer-s3 packages on my server.

Here, my implementation is divided into important parts.

 // SETUP var multer = require('multer'); var s3 = require('multer-s3'); var storage = s3({ dirname: 'user/uploads', bucket: auth.aws.s3.bucket, secretAccessKey: auth.aws.s3.secretAccessKey, accessKeyId: auth.aws.s3.accessKeyId, region: auth.aws.s3.region, filename: function (req, file, cb) { cb(null, Date.now()); } }); var upload = multer({storage: storage}).single('img'); // ROUTE module.exports = Router() .post('/', function (req, res, next) { upload(req, res, function (err) { if (err) { return res.status(401).json({err: '...'}); } return res.json({err:null,url: '..'}); }); }); 

What I want to do: convert the image before downloading it. I'm not sure if I need to use multer / busboy here, or I can just do it with NodeJS (this way I tagged NodeJS and expressed it as well).

So my question is: where can I intercept the download and convert it before loading into my S3 bucket?

Answer 1

Not sure if you are still looking for the answer to this question, but I had the same problem. I decided to expand the multer-s3 package.

I opened a request to transfer to the original repository , but for now you can use my plug .

Here is an example using the extended version:

  var upload = multer({ storage: multerS3({ s3: s3, bucket: 'some-bucket', shouldTransform: function (req, file, cb) { cb(null, /^image/i.test(file.mimetype)) }, transforms: [{ id: 'original', key: function (req, file, cb) { cb(null, 'image-original.jpg') }, transform: function (req, file, cb) { cb(null, sharp().jpg()) } }, { id: 'thumbnail', key: function (req, file, cb) { cb(null, 'image-thumbnail.jpg') }, transform: function (req, file, cb) { cb(null, sharp().resize(100, 100).jpg()) } }] }) }) 

EDIT: My fork is now also available through npm under the name multer-s3-transform .

Answer 2

I tried using the @ItsGreg fork, but couldn't get it to work. I managed to get this behavior using the standard multer-s3 configuration and inside the file upload endpoint, i.e.

app.post('/files/upload', upload.single('file'), (req, res) => {...})

I retrieve the file using request and passing a sharp buffer. The following works (and assumes you are using ~/.aws/credentials ):

  let request = require('request').defaults({ encoding: null }); let dataURI = `https://s3.amazonaws.com/${process.env.AWS_S3_BUCKET}/${image.defaultUrl}`; request.get(dataURI, function (error, response, body) { if (! error && response.statusCode === 200) { let buffer = new Buffer(body); const sizes = ['thumbnail', 'medium', 'large']; sizes.forEach(size => { sharp(buffer) .resize(image.sizes[size]) .toBuffer() .then(data => { // Upload the resized image Buffer to AWS S3. let params = { Body: data, Bucket: process.env.AWS_S3_BUCKET, Key: `${image.filePath}${image.names[size]}`, ServerSideEncryption: "AES256", }; s3.putObject(params, (err, data) => { if (err) console.log(err, err.stack); // an error occurred else console.log(data); // successful response }); }) }) } });