Capped Collection Performance Issues

Question

I am doing some tests to find out what bandwidth I can get from Mongodb. The documentation says that private collections are the fastest option. But I often find that I can write to a regular collection much faster. Depending on the exact test, I can often get twice as much bandwidth with a regular collection.

Am I missing something? How to fix this problem?

I have a very simple C ++ program that writes as many as 64,000 documents to the collection as quickly as possible. I record the total time and time expected for the database. If I do not change anything except the name of the collection, I see a clear difference between private and normal collections.

> use tutorial switched to db tutorial > db.system.namespaces.find() { "name" : "tutorial.system.indexes" } { "name" : "tutorial.persons.$_id_" } { "name" : "tutorial.persons" } { "name" : "tutorial.persons.$age_1" } { "name" : "tutorial.alerts.$_id_" } { "name" : "tutorial.alerts" } { "name" : "tutorial.capped.$_id_" } { "name" : "tutorial.capped", "options" : { "create" : "capped", "capped" : true, "size" : 100000000 } } > db.alerts.stats() { "ns" : "tutorial.alerts", "count" : 400000, "size" : 561088000, "avgObjSize" : 1402.72, "storageSize" : 629612544, "numExtents" : 16, "nindexes" : 1, "lastExtentSize" : 168730624, "paddingFactor" : 1, "systemFlags" : 1, "userFlags" : 0, "totalIndexSize" : 12991664, "indexSizes" : { "_id_" : 12991664 }, "ok" : 1 } > db.capped.stats() { "ns" : "tutorial.capped", "count" : 62815, "size" : 98996440, "avgObjSize" : 1576, "storageSize" : 100003840, "numExtents" : 1, "nindexes" : 1, "lastExtentSize" : 100003840, "paddingFactor" : 1, "systemFlags" : 1, "userFlags" : 0, "totalIndexSize" : 2044000, "indexSizes" : { "_id_" : 2044000 }, "capped" : true, "max" : 2147483647, "ok" : 1 } 

Linux version: 3.4.11-1.fc16.x86_64

mongo version: db version v2.2.2, pdfile version 4.5

This is a dedicated computer that does nothing but start the Mongodb server and my test client. For this test, the car is ridiculously crushed.

Answer 1

I see a problem. The web page above says that a limited "no index" collection will offer high performance. But...

http://docs.mongodb.org/manual/core/indexes/ says: "Before builds with version 2.2 there was no _id field. In 2.2, all private collections have a _id field, except for those that are in the local database."

I created another version of my test, which is written to a limited collection in the local database. Of course, there were no indexes in this collection, and my throughput was much higher!

Perhaps a review of closed collections at http://docs.mongodb.org/manual/core/capped-collections/ should clarify this point.

Answer 2

Cropped collections ensure that the insertion order is maintained. As a result, requests do not need an index to return documents when inserting an order. Without this overhead indexing, they can support higher insertion bandwidth.

According to the above definition, if you do not have any insertion of indexes into private collections, it should not be faster than insertion into a regular collection. Therefore, if you do not have any indexes, and if you have no other reason to go with a private collection, such as caching, showing the last n elements, I would suggest you go with regular collections.

Cropped collections ensure that the insertion order is identical to the disk order (natural order) and do so by prohibiting updates that increase the size of the document. Cropped collections allow only updating the size of the original document, which ensures that the document does not change its location on the disk.