node server crashes on object with over 1000 items

Question

node server crashes on object with over 1000 items

I have a collection with over 1000 items (Under 1000 items is ok.) which causes the following error and crash when requested via ajax call: (node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.

This does not occur on local development. (actually has nothing to do with local dev. I was running a pre v0.10.0 on local machine and this issue is post 10.0)

I am running node v0.10.8 with mongodb database and mongoose.

But what is best solution?

See code:

 Collection.find().select('_id', 'first_name').sort('startTime', -1).exec(function(err,                 

  docs){

            var data = {'aaData' : docs};
            res.send(data);

        });

node.js
mongodb
crash
mongoose

Answer 1

With that many docs in a result set, you should use Mongoose's support for streaming the result of the query rather than getting it in one big array.

var stream = Collection.find()
    .select('_id', 'first_name')
    .sort('startTime', -1)
    .stream();

stream.on('data', function (doc) {
  // do something with the doc like res.write(doc);
}).on('error', function (err) {
  // handle the error
}).on('close', function () {
  // the stream is closed, so complete the response with res.end();
});