Unable to get redis data from io socket

Question

I am creating real-time visualization using redis as a pubsub messenger between python and node. There always runs a python script that sets the redis hash with hmset. This side of the application works fine, if I enter the following example command: "HGETALL" sellers-80183917 "on the redis client, I get the correct data.

The problem is in the js side. I use the socketio and redis nodejs libraries to listen on a redis instance and publish the results on the Internet via d3js, namely:

I run the following code with node:

var express = require('express'); var app = express(); var redis = require('redis'); app.use(express.static(__dirname + '/public')); var http = require('http').Server(app); var io = require('socket.io')(http); var sredis = require('socket.io-redis'); io.adapter(sredis({ host: 'localhost', port: 6379 })); redisSubscriber = redis.createClient(6379, 'localhost', {}); redisSubscriber.on('message', function(channel, message) { io.emit(channel, message); }); app.get('/sellers/:seller_id', function(req, res){ var seller_id = req.params.seller_id; redisSubscriber.subscribe('sellers-'.concat(seller_id)); res.render( 'seller.ejs', { seller:seller_id } ); }); http.listen(3000, '127.0.0.1', function(){ console.log('listening on *:3000'); }); 

And this is the relevant part of the seller.ejs file that accepts user requests and displays information:

  var socket = io('http://localhost:3000'); var stats; var seller_key = 'sellers-'.concat(<%= seller %>); socket.on(seller_key, function(msg){ stats = []; console.log('Im in'); var seller = $.parseJSON(msg); var items = seller['items']; for(item in items) { var item_data = items[item]; stats.push({'title': item_data['title'], 'today_visits': item_data['today_visits'], 'sold_today': item_data['sold_today'], 'conversion_rate': item_data['conversion_rate']}); } setupData(stats); }); 

The problem is that the socket_on () method never gets anything, and I don’t see where the problem is, since everything seems to work fine except for this.

Answer 1

I think you might be confused as to what Pub / Sub really is in Redis. This is not a way to listen for hash changes; you can have a Pub / Sub channel named sellers-1 , and you can have a hash with the sellers-1 key, but they are not related to each other.

As described here :

Pub / Sub has nothing to do with key space.

There is a thing called key notifications , which can be used to listen for changes in key space (via Pub / Sub channels); however, this feature is not enabled by default because it will require more resources.

Perhaps a simpler way would be to post a message after the HMSET so that any subscribers know that the hash has changed (they would then retrieve the contents of the hash themselves, or the published message would contain the relevant data).

This leads us to the next possible problem: you only have one connection for subscribers, redisSubscriber .

From what I understand from the Node.js Redis driver, calling .subscribe() on such a connection will delete any previous subscriptions in favor of the new one. Therefore, if you have previously subscribed to sellers-1 and subscribed to sellers-2 , you will no longer receive messages from sellers-1 .

You can listen to several channels by passing an array of channels or passing them as arguments:

 redisSubscriber.subscribe([ 'sellers-1', 'sellers-2', ... ]) // Or: redisSubscriber.subscribe('sellers-1', 'sellers-2', ... ) 

You will obviously have to track every “active” seller’s subscription. Either this, or create a new connection for each subscription, which is also not ideal.

It is probably best to have one Pub / Sub channel on which all changes will be published, rather than a separate channel for each seller.

Finally: if your seller ID is not hard to guess (for example, if it is based on an incremental integer value), it would be trivial if someone wrote a customer who could listen to any seller they would like. This may not be a problem, but it is something to be aware of.