From many to many relationships with MongoDB on a large scale

I saw a lot of posts on how to do many-to-many relationships with MongoDB, but none of them mention scale. For example, these messages:

MongoDB Many-to-Many Association

How to organize many, many relationships in MongoDB

The problem I see with this setting is the limitation of the MongoDB 16MB document. Let's say I have user s, group s and post s. post have associated group and many user who may like this. A group has a lot of post and many user in it that can follow it. A user can have many favorite post and can follow many group s. If I built this with a relational database, I would install it like this:

 user: user_id username post: post_id group_id message group: group_id name post_likes: post_id liked_user_id group_followers: group_id follower_user_id 

Theoretically, a group can have a limited number of post after user s, a post can have an unlimited number of liked user s, and a user can have an unlimited number of liked post and group , which they follow if the correct page layout is performed in SQL queries.

How can I customize the MongoDB schema to achieve this scale?

+4
source share
2 answers

This is a good question that illustrates the problems with overwork and how to deal with it.

Example: Post like

Let’s follow the example of users who love messages, which is a simple example. Other relationships should be handled accordingly.

You are absolutely right that with the preservation of such messages inside the message, sooner or later it will lead to a problem when the size of very popular messages is reached.

So, you stepped back correctly to create the post_likes collection. Why do I call it right? Since it is suitable for your use cases and functional and non-functional requirements!

  • It scales indefinitely (well, there is a theoretical limit, but it is very large)
  • It is easy to maintain (create a unique index over post_id and liked_user_id ) and use it (both the user and the mail are known, so adding this is a simple insert or, most likely, upsert)
  • You can easily find out who likes this publication and which publication will appeal to those with whom users

However, I would expand the collection a bit to prevent unnecessary queries for some commonly used use cases.

Assume publication names and usernames cannot be changed. In this case, the following data model may make more sense.

 { _id: new ObjectId(), "post_id": someValue, "post_title": "Cool thing", "liked_user_id": someUserId, "user_name": "JoeCool" } 

Now suppose you want to display the username of all users who liked the post. With the model above, this will be one rather quick request:

 db.post_likes.find( {"postId":someValue}, {_id:0,user_name:1} ) 

If you save only identifiers, this rather common task will require at least two queries, and - given the restriction that there can be an infinite number of instances for memory consumption it is potentially huge (you'd need to store user identifiers in RAM).

Of course, this leads to some redundancy, but even when millions of people love mail, we are only talking about a few megabytes of relatively cheap (and easily scalable) disk space, getting more performance in terms of user convenience.

Now here's the thing: even if the usernames and message headers can be changed, you only need to make a few updates:

 db.post_likes.update( {"post_id":someId}, { $set:{ "post_title":newTitle} }, { multi: true} ) 

You trade, it takes some time to do some fairly rare things, such as changing a username or message for extreme speed for use cases that happen very often.

Bottom row

Keep in mind that MongoDB is a documented database. Thus, document the events of interest to you with the values ​​needed for future queries, and model your data accordingly.

+6
source

If you just store the relationship identifier inside reach collection arrays, you shouldn't have a big problem inside the same document. GridFS can be used, but usually more for media such as files, music, videos, etc. Using GridFS, you do a pain update.

0
source

All Articles