Geek Answers Handbook

Notification system (email, social networks)

Currently, I am writing a small internal platform for social networks for the company I work with, nothing big, ok. 40,000 users. This is basically something like Facebook or Google+, but with less features.

This platform has an email notification system (in addition to Ajax notifications for online users).

Now here is my problem - it’s just not scaling.

I have a mySQL table that stores all email notifications. Therefore, as soon as someone writes a story in a group, the system automatically inserts a row into this table, and the email_send (cronjob) function sends these letters to users (users can choose between instant, daily or weekly notifications)

So far so good - if the groups have few members.

But now create a group with 5k + members. As soon as the user submits the history to this group, it calls 5000 instances of SQL in the notification table.

How would you solve this? I thought that there is a desktop on the server that scans new stories / comments / material and runs the email_send function in the background. Would this scale be better? Or is there only a standard way to do this, and am I just thinking in the wrong direction?

Any point in the right direction will be greatly appreciated, thanks and hilarious xmas :)

//Marcus.

+4
source share
2 answers

You should not slow down your normal flow of web requests / responses by sending emails - just upload the data to the queue and let the background task take this work - does it look like you are already doing this with your cron right?

Is this the insert that causes the slowness? Then just normalize your data and just paste the groupID into your email queue table. This groupID is associated with a group that contains X users and their email addresses. So, you will insert 1 record instead of 5000. So very quickly, maybe something like this ...

notificationTable rowID | emailID | groupID | status groupTable groupID | groupName userTable userID | userName | emailAddress | groupID

If you want to do this even further, you can introduce a messaging system so that you can turn off simple (and fast) messages and have one or more listeners (on one or more separate servers) waiting for messages on one or more channels.

+3
source

Well, I came up with the following (possible) solution to my problem:

  • I already have an event table:

rowID (event_id)| story_id | initiator | group_id

  • create a new table not_notified ":

rowID | event_id

As soon as someone sends a new story / comment no matter what, the system will now insert one row into the "not_notified" table.

Every couple of minutes, the cron task starts (it can be run from another dedicated server if speed occurs here), and SELECT - the "not_notified" table. It simply iterates over all recordsets and searches for each event_id, pulls the history, checks the notification settings for notifying the user, and sends an email notification to the user. Finally, the script DELETE a line from "not_notified".

I probably don’t even need the "not_notified" table - this can be done with a simple "new" field in the event table, but an image of millions of records in this table ...

Thoughts?

0
source

More articles:


All Articles