Best way to store and retrieve large video files for a web application?

Question

I am making a video sharing site. So I thought about what is the best way to store a large number of video files. So it will be fast and efficient in terms of access to them.

I asked my colleague, they say that storing files in a folder with a hard drive will not be effective, as in many read / write operations this will be very slow.

Someone suggested storing mysql in blocks.

I'm confused here!

Let me know what you think?

Answer 1

Use your identification key in the database to sort the files in the folder.

Take the first number, put it in the first directory, the second number in the next .. and so on:

Example:

18

/ video / 1/8 / MyvideoFile

one thousand eight hundred ninety two

/ video / 1/8/9/2 / MyVideoFile

Alternatively, you can also sort them on different servers or NAS clusters, storing the server number in your database.

Hope this helps.

Answer 2

Usually storing large files in a database is not considered good practice. See this question and those related to it for further information.

Instead of just storing to disk, you might consider using a service like Rackspace Cloud Files or Amazon S3 if your budget allows.

Answer 3

Saving to mysql DB will be slower than saving binary files to your hard drive. You can store copies of files on multiple computers if one machine cannot handle the load, and use load balancing on the shelf to manage it.

Answer 4

A solution that requires little initial time on your part, but it saves you a lot of trouble in the long run ... uses a cloud storage service, for example:

http://aws.amazon.com/s3/ or http://www.rackspacecloud.com/cloud_hosting_products/files

It scales and your servers can focus on launching your web application.

Answer 5

The videos are too large to be effectively stored in MySQL. Use the file system. Come up with a smart way to sort video files in separate directories so that you don’t get into thousands of files in one directory. For example, all files starting with "a" go to the directory "a", etc.

If your site gets really big, it might make sense to upload your videos to a CDN like Amazon Cloudfront .

Answer 6

So, for example, you have a 50 GB file, you can write it to 50 disks in parallel, 1 GB on each disk, so you only need 1 second to write, and in the same way it takes 1 second to read from 50 disks . Consider file systems such as HDFS.