How many active streams can one node.js process handle

I need to monitor a bunch of twitter streams (using twitter's streaming API) in node.js. Which approach would be "better"?

  • Run multiple processes, 1 Process per Stream
  • Run 1 process that handles all of the streams

At this point I do not know how many streams I will have, but ideally the set up should be able to scale to 10000+ open streams.

Edit for more context: Most or all processing will be done by another server. Just looking to manage the streams in a reliable, maintainable and performant way.

The answer to this question depends a lot on

  1. How often the tweeters are tweeting
  2. What you are doing in processing the tweets

In particular, if the tweets are only coming in sporadically, like 1 tweet per user every 3+ minutes, then I think 1 instance of node should be fine for handling all of them.

  • Run single process, measure cpu load. If one cpu is not enough, then run as much processes as CPU cores, try to distribute streams evenly between processes.