Async calling out generated functions in series

Question

Async calling out generated functions in series

I am having problem with calling the generated functions in serial. I am using the async library and the code seems to work when there is no deep callback calling needed. When I add the real scenario it throws errors.

Here is the example which works, returns the array of 0 to 4:

Scrape.prototype.generatePageFunctions = function() {
  var functionList = new Array(), self = this;

  for (var i = 0; i < this.pageSet; i++) {
    (function(i) {
      functionList.push(function(cb) {
        // Inner functions which will be called in seriers
        var timeoutTime = parseInt(Math.random() * 5000 + 3000, 10);

        setTimeout(function() {
          self.setIndex(i);
          //self.getSite(function)

          cb(null, i);
        }, timeoutTime);
      });
    })(i);
  }
  return functionList;
}

Scrape.prototype.run = function() {
  var functionList = this.generatePageFunctions();

  async.series(functionList, function(err, results) {
    console.log('Job is completed ');
    console.log(results);
  });
}

Now adding the real scenario like downloading the site and then put in callback:

Scrape.prototype.generatePageFunctions = function() {
  var functionList = new Array(), self = this;

  for (var i = 0; i < this.pageSet; i++) {
    (function(i) {
      functionList.push(function(cb) {
        // Inner functions which will be called in seriers
        var timeoutTime = parseInt(Math.random() * 5000 + 3000, 10);

        setTimeout(function() {
          self.setIndex(i);
          self.getSite(function(result) {
            // Async callback to pass the data
            cb(null, result);
          });
        }, timeoutTime);
      });
    })(i);
  }
  return functionList;
}

Error is like this, even if passing instead of result iterator variable i:

/home/risto/scrape/node_modules/async/lib/async.js:185
            iterator(x.value, function (err, v) {
                      ^
TypeError: Cannot read property 'value' of undefined
    at /home/risto/scrape/node_modules/async/lib/async.js:185:23
    at /home/risto/scrape/node_modules/async/lib/async.js:108:13
    at /home/risto/scrape/node_modules/async/lib/async.js:119:25
    at /home/risto/scrape/node_modules/async/lib/async.js:187:17
    at /home/risto/scrape/node_modules/async/lib/async.js:491:34
    at /home/risto/scrape/scraper/scrape.js:114:13
    at /home/risto/scrape/scraper/scrape.js:64:16
    at Object.<anonymous> (/home/risto/scrape/scraper/engines/google.js:58:12)
    at Function.each (/home/risto/scrape/node_modules/cheerio/lib/api/utils.js:133:19)
    at [object Object].each (/home/risto/scrape/node_modules/cheerio/lib/api/traversing.js:69:12)

// Edit

Only result which gets added into the complete callback is the first one, other functions are never called. Also for information the functions return object literals if that does matter.

javascript
node.js
asynchronous

Answer 1

There is nothing wrong with your code. Creating a simple testcase shows that.

I created a mock:

Scrape = function() {
  this.pageSet = 5;
}

Scrape.prototype.setIndex = function() {
}

Scrape.prototype.getSite = function(cb) {
  cb('works');
}

and calling the run method it outputs the expected:

[ 'works', 'works', 'works', 'works', 'works' ]

So the problem is somewhere else. Have you tried to check the functionList variable in your run method?

Answer 2

Thank you @KARASZI István, all the code above is correct, the problem seemed in somewhere else. The deepest callback got called multiple times but the outer one got called only once.