Downloading files through the application for the playback platform on S3 without “touching” the disk

Question

I need to implement a server that allows some users to upload, possibly huge files (several gigabytes) of files to S3 (they don’t know that it is S3, though). I already implemented something similar in Python and a basic implementation using the Play platform.

My download function now looks like this:

public Result uploadFile() { List<Http.MultipartFormData.FilePart> files = request().body().asMultipartFormData().getFiles(); if (CollectionUtils.isNotEmpty(files)) { Bucket bucket = MY_BUCKET; UUID timeuuid; Promise<UploadResult> promise; ObjectNode responseMessage = Json.newObject(); for (Http.MultipartFormData.FilePart file : files) { timeuuid = TimeUUID.now(); promise = Promise.promise(new UploadFunction(timeuuid, bucket, file.getFile(), obtainS3Connection())); responseMessage.set(file.getFilename(), TextNode.valueOf(timeuuid.toString())); } return accepted(responseMessage); } else { return badRequest("files empty"); } } 

I am not worried about the most loaded code , and actually I have implemented it many times. I worry about making it so playable! it doesn’t save files to disk and that the sent files “pass through” my server without writing to the disk (even as temporary files). I have some kind of processing and once a “client” encryption of some content, I can do it as a stream.

I was wondering if Play already does this? Or how can I achieve this?

I assume that Play (and Django in my previous version of this service):

 Upload --> Play MyProcessing -------> S3 \ / \ / Disk 

What I would like to do:

 Upload --> Play --> MyProcessing -------> S3 

The reason I want to do this is because I want me to be able to deploy my service on instances with very small disks, and I also want to ensure that no one can find the unencrypted file (if I encrypt it) after downloads.

UPDATE : I'm not giving up ... for now.

I found several possible solutions that unfortunately use Scala. They all revolve around the same topic as the discussion: Play 2.x: Download the active file using Iteratees

I was wondering if this is possible using Java (I am the only one in my company who knows Scala, so I cannot deploy something for which I would be the only person who could perform maintenance ... because sometimes I want to take breaks or engage in other projects).

Answer 1

I never did this with Play, but I did something similar in Java. The path to success for us was to send files directly to S3. Basically, you create a download form for a user where the message URL is not provided by your service, and Amazon and the user browser are directly loaded onto S3.

Upon successful loading, S3 will then return an empty document with a status code of 200 when the object is successfully loaded. In our case, we make a mail call through jQuery and wait for the jQuery handler to succeed in this post, and only after that we update our database. Alternatively, you can specify the redirect form field with the URL where the user is redirected after a successful upload.

Details on this can be found here: http://doc.s3.amazonaws.com/proposals/post.html