MongoDB: How to create Twitter-style proponents / next relationship model in MongoDB?

Question

I am developing a mobile application for Android that has potentially many users (say, about 1 million). These users can follow other users (e.g. Twitter). The application synchronizes user data through a remote REST server. The user data itself is stored in a document-oriented database (in my case, it is MongoDB).

I am currently asking myself the best way to design a user model, including its follower and the following relationships. The first thought was to embed relationships in a user document.

Example user document:

{ "_id":"50fd6bb530043e3c569af288", "name":"Marsha Garcia", "follower"["50fd6bb530043e3c569af287","50fd6bb530043e3c569af289","50fd6bb530043e3c569af28c"], "following":["70fd6bb530043e3c569af289","10fd6bb530043e3c569af222","89fd6bb530043e3c569af45o"] } 

It is positive that the following / next relationships are already connected to the user. However, let's say that a user follows about 100,000 or more other users. Then the size of the document will become very large. If I load this user object through the REST service in my mobile application, this may take some time. In addition, in the worst case scenario, a user document may exceed the MongoDb document limit by 16 MB.

So my second thought was to model the follower and the following relationships in a more classic way: an additional document containing the following relationships of each user.

Example document 'user relation':

 { "_id": 50fe65828de290c0a8a8ea2d" "uid": "50fd6bb530043e3c569af288", "rel_uid": "50fe65828de290c0a8a8e9a6", "type": "FOLLOWING" } 

The positive thing is that the size of each user document will remain constant. The downside is that with a lot of users and the following relationships, I could easily get millions of entries in my MongoDB user relationship collection. Of course, I'm going to set the margin index, but I'm not quite sure if this solution will scale very well in relation to the case of using an application user requesting his / her current subscribers.

I would appreciate any thoughts, impressions about my modeling problem. Perhaps someone even has a better approach to the solution.

thanks in advance.

Answer 1

 1. collection users: - userid - username - userpass - other user specific info user 2. collection following: - userid - [array of followingid] 3. collection followed: - userid - [array of followedid] 4. messages_relation collection: - userid - messageid - time 5. messages_text: - messageid - text 

Answer 2

I would start by reading this documentation on storing comments in the CMS, if you haven't already. Although there is one and the same common problem for comments - if you cannot store all comments in one document (in your case, followers / next).

Any Hybrid approach (which uses fewer documents and retains some relationships within the same document) or the approach you describe should work well.

I also suggest creating a simple POC to test search performance, etc. It might make sense to cache some results or precompile them. As a rule, this is normal in such systems if everything is not immediately agreed upon for all users (for example, immediately if the follow-up counter is correct).

This is probably not an ideal solution and may require several solutions for optimal performance (for example, how the user and followers are processed), as the number of followers increases, for example).

Answer 3

you can check the flockDB database in which the compatibility lists are stored. https://github.com/twitter/flockdb