Passing data into node.js callbacks - the asynchronous issue

Question

Passing data into node.js callbacks - the asynchronous issue

Take the code below, in which an object data is initialized with some values that are then processed by some time-intensive function, like database access. If the function succeeds, the name of the successful data item is printed to the console. Otherwise, a failure notice is printed:

data = {first:  'someinfo',   second:  'somemoreinfo',   third:  'evenmoreinfo'};

for (var item in data) {
    timeIntensiveFunction(item, data[item], function(err) {
        if (!err) {
            console.log(item + ' processed successfully');
        } else {
            console.log(item + ' failed');
        }
    });
}

You would expect the console to show this, assuming the function succeeds for all three data items:

first processed successfully
second processed successfully
third processed successfully

Instead it will show this, assuming the first database access takes longer than the for loop:

third processed successfully
third processed successfully
third processed successfully

This is because the console logging is done in a callback, which would reasonably only be called after the for loop is done, because timeIntensiveFunction() takes so long. By the time the first callback is called, item already has its last value, third.

How do you pass the 'current' value of item into the callback?

javascript
node.js
asynchronous
callback

Answer 1

Problem is because it's calling the callback with the last item only.

You can bind each of your item with a function like bellow.

var printStatus = function(item){
    return function(err) {
        if (!err) {
            console.log(item + ' processed successfully');
        } else {
            console.log(item + ' failed');
        }
    }
}

for (var item in data) {
    timeIntensiveFunction(item, data[item], printStatus(item));
}

Answer 2

This is a common "gotcha" with closures in javascript. One way around it is to wrap your function call in an anonymous function and rescope item. Like so:

for (var item in data) {
    (function(item){
        timeIntensiveFunction(item, data[item], function(err) {
           if (!err) {
               console.log(item + ' processed successfully');
           } else {
               console.log(item + ' failed');
           }
        });
   })(item);
}

Answer 3

If you're looking for a library to make working with async tasks more easy, check out caolan/async.

var async = require("async");

var data = [{id: "first"}, {id: "second"}, {id: "third"}];

function timeIntensiveFunction(item, done) {
  // do something
  console.log("time intensive task started:", item.id);

  // err?
  // if (err) return done(err);

  done();
}

function processItem(item, done) {
  timeIntensiveFunction(item, function(err) {
    if (err) return done(err);
    console.log("task complete:", item.id);
    done();
  });
}

async.map(data, processItem);

Output

time intensive task started: first
task complete: first
time intensive task started: second
task complete: second
time intensive task started: third
task complete: third

For users looking to learn how to do this without a library, you can see the revision history of this answer.