Geek Answers Handbook

MongoDB - Saving files in a database or external storage?

We are using MongoDB. In our application, we need to store several files per user. Our application will be deployed on AWS. I thought of using one of the following options for storing files -

  • Store files in MongoDB. I know that MongoDB has a limit on the maximum document size of 16 MB, but most likely our files will not go beyond that.
  • Use MongoDB GridFS to store files.
  • Use Amazon S3 to store files, and links to these files will be processed in MongoDB.
  • Using local EBS file systems / volumes in AWS.

Which of the above methods works best? As for performance, scalability? Which approach will be more scalable? I also want to use CDN to cache files. My preference is for AWS S3, since I can use CDN to cache files, my file storage will be agrostatic. Also, the size of my database will not increase significantly, since I store files outside the database.

+8
mongodb
source share
2 answers

I think the best options here are GridFS and S3; I myself will go with the latter. Click the file on S3, and then save the name and file of the bucket in your Mongo document. If your business or query requirements are not such that all the data should be present in the document, I think this is the best way to go.

I used this solution in production and is very easy to scale. There is little impact on your Mongo collection, and you don’t have to worry about storing a huge amount of data. Just save the key and let S3 take care of all this. You can always store them somewhere even later, since your system is rather aggregated for storage.

+6
source share

At first I will use only the local file system in development mode, during production I will use GridFS or Amazon S3.

Allows you to make this more understandable.

First point (Store files in MongoDB itself.)

  • If you process images as small or as small as 16 MB, you should store them in your collections.

contrast

Every time you make a request, you should know that you are exploring the entire collection, so it will take a bit (you can exclude the “image” field to avoid this.)

Second point (use MongoDB GridFS to store files.)

  • If you process images larger than (16 MB>)
  • You can add metadata (I like it).

Take a look at the GridFs Docs .

This article talks about the pros and cons of using GridFs , it can also be helfup

also When should I use gridfs?

Third Point (Use Amazon S3)

I am not very familiar with S3 (I never use it).

But this is from Amazon docs when should I use Amazon S3?

S3 is free to join, and it is a pay-as-you-go service, which means that you only ever pay for any hosting and bandwidth costs that you use, which makes it very attractive for start-ups, flexible and thrifty companies to minimize costs.

In addition, Amazon's fully scalable, fast and reliable service makes it very attractive to video makers and marketers around the world.

Amazon offers S3 as a hosting system, with a price that depends on the geographic location of the data center where you store your videos.

Four points (use local file system)

I only use the file system, when I test my applications, I never use it in production, since with my POV, it is not so scalable.

In my personal opinion, I would use GridFS, but I think that you need to analyze the requirements of your application and therefore know which storage adapter is using

+2
source share

More articles:


All Articles