Asynchronous callbacks in a meteor app

Question

Asynchronous callbacks in a meteor app

I'm using the ntwitter node.js module to access twitter's streaming API from inside a meteor app, but when trying to insert into a collection inside the callback function the app crashes:

twitter.stream('statuses/filter', {'track':'whatever'}, function(stream) {
  stream.on('data', function (data) {
      //logging the data coming back works fine
      console.log(data);
      //the next line throws "Error: Meteor code must always run within a Fiber"
      Tweets.insert(data); 
  });
});

Is there a recommended approach to using async callbacks in the context of Meteors linear execution model? I tried wrapping the insertion inside a new Fiber which seems to work but I'm unsure of any implications it may have.

I found this http://gist.io/3443021 which was helpful but I'm still not sure which approach would suit my particular case so any help would be appreciated.

Cheers

node.js
twitter
meteor

Answer 1

We used a different design pattern. In the async callback, we act more like a device driver and simply buffer the result in memory:

var tweets = [];

twitter.stream('statuses/filter', {'track':'whatever'}, function(stream) {
  stream.on('data', function (data) {
      //logging the data coming back works fine
      console.log(data);
      //the next line throws "Error: Meteor code must always run within a Fiber"
      tweets.push(data); 
  }); 
});

and then later on, back within the normal Meteor execution environment within a Fiber, either in a timer or a result of a function, we drain the tweets array and do the insert then. A Javascript array doesn't care whether it is running within a Fiber or not.

In our case, we are doing this with asynch IMAP email rather than tweets, but the analogy still holds.

Answer 2

Wrap your callback in Meteor.bindEnvironment like so:

twitter.stream('statuses/filter', {'track':'whatever'}, function(stream) {
  stream.on('data', Meteor.bindEnvironment(function (data) {
      //logging the data coming back works fine
      console.log(data);
      //the next line throws "Error: Meteor code must always run within a Fiber"
      Tweets.insert(data); 
  }));
});

According to this SO post about Async wrappers on the Meteor Server, you want to use Meteor.bindEnvironment when you are managing a callback using a third party api/npm module (which looks to be the case)

Meteor.bindEnvironment creates a new Fiber and copies the current Fiber's variables and environment to the new Fiber.