Why doesn't Mongoose save records as script executes?

Question

Why doesn't Mongoose save records as script executes?

I'll try to be as terse as possible - Mongoose doesn't seem to save records until the entire script ends, even when the save() portion is called asynchronously:

var reallyBigData = require('./data.json');

var importData = function(data) {
  var newObj = new Obj; // Mongoose Object
  newObj.x = data.x;
  newObj.save();
};

this.on('importData', importData);

for(var i in reallyBigData) {
  this.emit('importData', reallyBigData[i]);
};

This is a gross simplification, but you get the idea. As I run the script and query Mongo, I don't see any records as the script is running. If I limit the number to say, 30, and import those it finishes fast and I see the records afterwards.

Why isn't Mongoose saving each record as the script is executing?

node.js
mongodb
mongoose

Answer 1

The issue is that x.save() (or newObj.save()) is asynchronous and you don't have anything waiting for it to finish other than Node itself.

Being asynchronous, it only starts the task of updating the collection in MongoDB and exits immediately after. It won't actually be "done" until later, when an optional callback would be called:

x.save(function (err, instance) {
    // at this point, it either erred or saved
});

Though, it is possible for the x.save() to finish nearly instantly. The lower number of records finishing in time to be seen means you have a race condition -- which will finish first: the queries to MongoDB or Node/V8 executing the block of code?

And, using an EventEmitter, at least how you have it sampled here, isn't enough to make your loop asynchronous since you're still emiting events synchronously and emit() will loop through any handlers synchronously.

However, there are numerous "control flow" libraries available that can help. These include implementations of promises, futures, and iterators like async:

var reallyBigData = require('./data.json');

async.each(Object.keys(reallyBigData),
    function (key, callback) {
        new Obj({ x: reallyBigData[key] }).save(callback);
    },

    function (err, results) {
        // called with an `err` or an `Array` of `results` (saved `Obj`s)
    }
);

Answer 2

First off I think you have a nasty error here:

var importData = function(data) {
  var newObj = new Obj; // Mongoose Object
  Obj.x = data.x;
  x.save();
};

Which should most likely read as:

var importData = function(data) {
  var newObj = new Obj; // Mongoose Object
  newObj.x = data.x;
  newObj.save();
};

Since it looks like you are trying to operate on your schema and not on the model that you "newed" out of the schema. Does that make sense? Or perhaps that is how you have it and you just miss typed? As is I don't see how you will be getting the expected results from the code that you have in place, regardless of how it is executing to the database. Perhaps as you have it now it is struggling to detect that the model has changed values, and or it is failing your validation.