Node.js, Socket.io, Redis pub / sub high volume, low latency difficulties

Question

When combining socket.io/ node.js and redis pub / sub in an attempt to create a real-time webcasting system driven by server events that can handle multiple transports, there are apparently three approaches:

• 'createClient' connection redis and subscribe to channels (channels). When connecting the socket.io client, attach the client to the socket. In the redis.on event ("message", ...) call io.sockets.in (room) .emit ("event", data) to distribute all clients in the corresponding room. How to reuse a redis connection in socket.io?

• The 'createClient' connection is redis. To connect to the socket.io client, connect the client to the socket.io room and subscribe to the corresponding redis channels (channels). Include redis.on (“message”, ...) in closing the client’s connection and upon receiving the client’s call client.emit (“event”, data) in order to raise an event on a specific client. As an answer in RedisStore Usage Examples in socket.io

• Use a RedisStore baked in socket.io and 'broadcast' from a single send channel in Redis, following the socketio-spec protocol.

Number 1 allows you to process one and the other Redis event once for all clients. Number 2 offers a more direct hook at Redis pub / sub. Number 3 is simpler but has little control over messaging events.

However, in my tests, everyone demonstrates unexpectedly poor performance with more than one connected client. The server events in question are 1,000 messages posted to the redis channel as quickly as possible so that they can be distributed as quickly as possible. Performance is measured by timings on connected clients (based on socket.io-client based on log time data in the Redis list for analysis).

What I suppose is that in option 1, the server receives the message, and then writes it to all connected clients sequentially. In option 2, the server receives each message several times (once per client subscription) and writes it to the corresponding client. In any case, the server does not fall into the second message until it is transmitted to all connected clients. The situation is clearly exacerbated by the growth of concurrency.

This seems to contradict the perceived wisdom of the stack capabilities. I want to believe, but I'm afraid.

Is this scenario (low latency distribution of large volumes of messages) just not an option with these tools (yet?), Or did I miss the trick?

Answer 1

I thought that was a reasonable question, and he studied it for a while. I spent a bit of time looking for examples that you can get from some useful tips.

Examples

I like to start with direct examples:

• sample code light im
• Node.js + Redis Pub / Sub + socket.io demo

The light pattern is one page (note that you will want to replace redis-node -client with something like Matt Rannie's node_redis :

/* * Mclarens Bar: Redis based Instant Messaging * Nikhil Marathe - 22/04/2010 * A simple example of an IM client implemented using * Redis PUB/SUB commands so that all the communication * is offloaded to Redis, and the node.js code only * handles command interpretation,presentation and subscribing. * * Requires redis-node-client and a recent version of Redis * http://code.google.com/p/redis * http://github.com/fictorial/redis-node-client * * Start the server then telnet to port 8000 * Register with NICK <nick>, use WHO to see others * Use TALKTO <nick> to initiate a chat. Send a message * using MSG <nick> <msg>. Note its important to do a * TALKTO so that both sides are listening. Use STOP <nick> * to stop talking to someone, and QUIT to exit. * * This code is in the public domain. */ var redis = require('./redis-node-client/lib/redis-client'); var sys = require('sys'); var net = require('net'); var server = net.createServer(function(stream) { var sub; // redis connection var pub; var registered = false; var nick = ""; function channel(a,b) { return [a,b].sort().join(':'); } function shareTable(other) { sys.debug(nick + ": Subscribing to "+channel(nick,other)); sub.subscribeTo(channel(nick,other), function(channel, message) { var str = message.toString(); var sender = str.slice(0, str.indexOf(':')); if( sender != nick ) stream.write("[" + sender + "] " + str.substr(str.indexOf(':')+1) + "\n"); }); } function leaveTable(other) { sub.unsubscribeFrom(channel(nick,other), function(err) { stream.write("Stopped talking to " + other+ "\n"); }); } stream.addListener("connect", function() { sub = redis.createClient(); pub = redis.createClient(); }); stream.addListener("data", function(data) { if( !registered ) { var msg = data.toString().match(/^NICK (\w*)/); if(msg) { stream.write("SERVER: Hi " + msg[1] + "\n"); pub.sadd('mclarens:inside', msg[1], function(err) { if(err) { stream.end(); } registered = true; nick = msg[1]; // server messages sub.subscribeTo( nick + ":info", function(nick, message) { var m = message.toString().split(' '); var cmd = m[0]; var who = m[1]; if( cmd == "start" ) { stream.write( who + " is now talking to you\n"); shareTable(who); } else if( cmd == "stop" ) { stream.write( who + " stopped talking to you\n"); leaveTable(who); } }); }); } else { stream.write("Please register with NICK <nickname>\n"); } return; } var fragments = data.toString().replace('\r\n', '').split(' '); switch(fragments[0]) { case 'TALKTO': pub.publish(fragments[1]+":info", "start " + nick, function(a,b) { }); shareTable(fragments[1]); break; case 'MSG': pub.publish(channel(nick, fragments[1]), nick + ':' +fragments.slice(2).join(' '), function(err, reply) { if(err) { stream.write("ERROR!"); } }); break; case 'WHO': pub.smembers('mclarens:inside', function(err, users) { stream.write("Online:\n" + users.join('\n') + "\n"); }); break; case 'STOP': leaveTable(fragments[1]); pub.publish(fragments[1]+":info", "stop " + nick, function() {}); break; case 'QUIT': stream.end(); break; } }); stream.addListener("end", function() { pub.publish(nick, nick + " is offline"); pub.srem('mclarens:inside', nick, function(err) { if(err) { sys.debug("Could not remove client"); } }); }); }); server.listen(8000, "localhost"); 

Documents

There is a ton of documentation, and apis are quickly changing to this type of stack, so you have to weigh the time relevance of each document.

• node activity streams
• casting sheet example
• like node redis pubsub
• redis latency
• redis cookbook Using Pub / Sub for Asynchronous Communication
• link general tips
• node redraw bindings
• google groups nodejs question

Matters Related

Just a few related questions, this is a hot topic on the stack:

• Redis pub / sub for chat server in node.js
• How to create redis pub / sub for instant messaging?

Famous Tips (ymmv)

Disable or optimize pooling, use effective bindings, monitor latency and make sure that you are not duplicating work (i.e. do not need to publish to all listeners twice).