Why does MongoDB take up so much space?

Question

I am trying to store records with a set of doubles and ints (about 15-20) in mongoDB. Records mainly (99.99%) have the same structure.

When I store data in root , which is a very structured data storage format, the file is around 2.5 GB for records of 22.5 million . However, for Mongo, the database size (from the show dbs command) is about 21 GB , while the data size (from db.collection.stats() ) is 13 GB .

This is a huge overhead ( Clarify: 13GB vs 2.5GB, I’m not even talking about 21GB ), and I think this is because it stores both keys and values , So the question is why and how Mongo does not do better reducing it?

But the main question: what is the impact of performance on this? I have 4 indexes, and they go beyond 3 GB , so running the server on one 8-gigabyte computer can be a problem if I double the amount of data and try to save a large working set in memory.

Any guesses if I should use SQL or some other DB? or maybe just keep working with ROOT files if anyone tried them?

Answer 1

Basically, this mongo is preparing to enter data. Mongo pre-localizes the data store to prevent (or minimize) disk fragmentation. This pre-location is observed as a file created by the mongod instance.

First, it creates a 64 megabyte file, the next 128 MB, the next 512 MB, and then until it reaches 2 GB files (the maximum size of the previously allocated data files).

There is something else Mongo can do to use more disk space, such as journaling ...

For much more information on how mongoDB uses storage space, you can take a look at this page in a specific section called Why do the files in my data directory have more data in my database?

There are a few things you can do to minimize the space that is used, but these tools (for example, using the --smallfiles ) are usually recommended for development and testing - never for production.