Async and recursion in nodejs

Question

Async and recursion in nodejs

Beginning with express and mongoose i often need to do some batch operations on collections. However it usually involves callbacks which is a pain given how concurrency is coded in nodejs. so basically

//given a collection C 
var i = 0;
var doRecursive = function(i){
    if(i<C.length){
      C[i].callAsync(err,result){
        i=+1;
        return doRecursive(i);
       }
    }else{
      return done();
    }
}
doRecursive(i);

Now i dont remember what is the max stack before i get a stackover flow with node , but i guess with 10 000 elements , it wont do. I wonder if there are other ways to handle this, if yes , what are they? thanks

javascript
node.js
mongodb
express
mongoose

Answer 1

If the goal is to iterate an collection asynchronously, there are numerous control flow libraries available.

A good example is async and its reduce function:

async.reduce(C, 0, function (memo, item, callback) {
    item.callAsync(function (err, result) {
        if (err) {
            callback(err);
        } else {
            callback(null, memo + result);
        }
    });
}, function (err, result) {
    // ...
});

Note: It's not entirely clear what value you wanted to get from doRecursion, so this just uses addition for an example.

Answer 2

i think you can simply self-iterate instead of true recursion, since you're not drilling into a deep object:

function doRecursive (C, i){
    i=i||0;
    if(i<C.length){
       C[i].callAsync(err, function(result){
          doRecursive(C, ++i);
       });
    }else{
       done();
    }
};

doRecursive(C);

this does not create a tall stack if the code functions as labeled. i localized C so that it executes faster and is potentially re-usable on other collections. the pattern also makes it easy to defer it for long-running operations, just by changing

doRecursive(C, ++i);

to

setTimeout( doRecursive.bind(this, C, ++i), 50 );