Geek Answers Handbook

Filepicker? Upload large files via HTML5 to S3 without backend

Uploading files using multipart / form-data is straightforward and takes a very long time until you start focusing on uploading large files. If we carefully consider what happens during file upload:

  • the client sends a POST request with the contents of the file to BODY

  • the web server accepts the request and initiates data transfer (or returns error 413 if the file size exceeds the limit)

  • webserver begins to fill buffers (depending on the size of files and buffers), store it on disk and send it through a socket / network to the server

  • back-end authenticates (see as soon as the file is uploaded)

  • the back-end reads the file and cuts several headers Content-Disposition, Content-Type, saves it again on disk back-end does everything you need to do with the file

To avoid such overhead, we upload the file to disk (Nginx client_body_in_file_only) and manage the callback for further sending along the line. Then the queue worker selects the file and does everything necessary. It works for interservice communication quite smoothly, but we must solve a similar problem with client-side downloads.

We also have a client solution for downloading S3. No interaction occurs. To download the video, we manage the video to convert to h.264 Baseline / AAC format with Zencoder.

We are currently using a modified s3-swf-upload-plugin- based Flash downloader with the Zencoder JS SDK combination, which is really effective but uses Flash.

Question. How to achieve the same goal with HTML5 file loader? Filepicker.io and Zencoder solve the problem? What is the recommended way to control the loading of an HTML5 file without using external interaction?

The requirements are as follows:

  • HTML5, not flash
  • to download the video, followed by processing, to make it compatible with HTML5 players and a mobile phone.
  • for loading images with subsequent processing (resizing, cropping, rotation)
  • for downloading documents such as PDF with preview function

Does https://www.filepicker.com comply

+7
javascript html5 upload amazon-s3
source share
3 answers

The requirements are as follows:

HTML5, not flash

Filepicker now supports a full responsive widget that is pure html and css.

to download the video, followed by processing, to make it compatible with HTML5 players and mobile

Filepicker now offers to transcode most video formats to h264 and webm for playback on mobile devices. https://www.filepicker.com/documentation/file_processing/video_conversion/video

for loading images with subsequent processing (resizing, cropping, rotation)

Filepicker offers cropping and rotation in new widgets, as well as resizing, sharpening and watermarking through the API.

for downloading documents such as PDF with preview function

We offer the ability to convert from 19 different file formats to many output formats. https://www.filepicker.com/documentation/file_processing/document_conversion/document

+1
source share

I have been using filepicker for 2 years now and no doubt it is worth it. don't try to control file downloads (from google drive, from ios, from my camera, from Dropbox ...) Filepicker does a great job of this and provides you with a ready-to-use url. Spend more time working with your core business; uploading files is really easy to delegate

+1
source share

To upload large files to S3, there is a REST API for loading multi-page files , which works as follows

  • initiate download
  • Loading file contents into multiple requests
  • complete download

The API is also available for calling from javascript, and the downloaded file can be divided into several requests using the / Blob file API

The only problem is to be able to authenticate S3 from javascript, you need to pass your authentication data. This is usually solved by some intermediate layer, such as PHP, so authentication data is not stored in javascript files.

A similar SO question: HTML5 and Amazon S3 Downloading multiple parts

EDIT

  • image operations, such as cropping and resizing, can be done using the canvas. Just load the local image into the canvas element, make the necessary changes and then create the jpeg / png image data stream using the canvas.toDataUrl method.
  • PDF preview is possible, there is a PDF.js lib that can display a local PDF file on canvas
  • AFAIK there is no way to do client side video conversion
0
source share

More articles:


All Articles