I'm using Node.js to loop through what could eventually be a pretty big array of posts.
If I were doing something similar using client side JavaScript, I would use timers as explained here so as not to block the thread.
My Question is: "Is still a sound practice server side?" or "Should I approach the problem differently?"
The proper way to do that in node.js is to break up your work into chunks and use process.nextTick
to queue the next chunk once the current one has completed. That way you allow other queued callbacks to be executed between each chunk of work.
UPDATE: as of Node.js 0.10, setImmediate
should typically be used instead of process.nextTick
for this purpose as setImmediate
yields to the event loop to make sure I/O is not being starved, but process.nextTick
doesn't.
JohnnyHK's suggestion is valid. I would consider web workers when the task can be completed later (ie: Added to a queue).
Take a look to async library. It provides the async version of the usual collection methods (map, filter, each...). AFAIK, processing your array asynchronously won't block other processes nor users.