node.js redis async query

Question

node.js redis async query

Hope someone can assist with a (simple) async question on node-redis. I'm trying to load a set from a hash in the redis db and then use that populated set further on. Here's the code snippet :-

var redis_client = redis.createClient(REDIS_PORT, REDIS_URL); 

redis_client.hgetall(target_hash,function(e,o){

    Object.keys(o).forEach(function(target){

        // get the "name" from the hash         
        redis_client.hget(o[target],"name",function(e,o){
        if (e){
                 console.log("Error occurred getting key: " + e);
              }
        else {
                 redis_client.sadd("newset",o);
             }
    });

});

// the following line prints nothing - why ?? 

redis_client.smembers("newset",redis.print);

When I examine the contents of "newset" in redis it is populated as expected, but at runtime it displayed as empty. I'm sure it's an async issue - any help much appreciated !

node.js
redis

Answer 1

hgetall is an asynchronous call: when it receives a reply from the redis server, it will eventually call your callback function (target) { ... }. But within your script, it actually returns immediately. Since hgetall returns very fast, Node will immediately run the next statement, smembers. But at this point the sadd statements haven’t run yet (even if your system is very fast because there hasn’t been a context switch yet).

What you need to do is to make sure smembers isn’t called before all the possible sadd calls have executed. redis_client provides the multi function to allow you to queue up all the sadd calls and run a callback when they’re all done. I haven’t tested this code, but you could try this:

var redis_client = redis.createClient(REDIS_PORT, REDIS_URL);

redis_client.hgetall(target_hash, function(e, o) {
  var multi = redis_client.multi();
  var keys = Object.keys(o);
  var i = 0;

  keys.forEach(function (target) {
    // get the "name" from the hash     
    redis_client.hget(o[target], "name", function(e, o) {
      i++;
      if (e) {
        console.log("Error occurred getting key: " + e);
      } else {
        multi.sadd("newset", o);
      }

      if (i == keys.length) {
        multi.exec(function (err, replies) {
          console.log("MULTI got " + replies.length + "replies");
          redis_client.smembers("newset", redis.print);
        });
      }
    });
  });
});

Some libraries have a built-in equivalent of forEach that allows you to specify a function to be called when the loop is all done. If not, you have to manually keep track of how many callbacks there have been and call smembers after the last one.

Answer 2

You shouldn't use multi unless you need actually need a transaction.

just keep a counter of the transactions and call smembers in the final callback

var redis_client = redis.createClient(REDIS_PORT, REDIS_URL); 
var keys = Object.keys(o);
var i = 0;
redis_client.hgetall(target_hash,function(e,o){

    Object.keys(o).forEach(function(target){

        // get the "name" from the hash         
        redis_client.hget(o[target],"name",function(e,o){
        i++
        if (e){
                 console.log("Error occurred getting key: " + e);
              }
        else {
                 redis_client.sadd("newset",o);
                 if (i == keys.length) {
                    redis_client.smembers("newset", redis.print);
                 }

         }});