FYI: I am doing this already with Web Workers, and it works fine, but I was just exploring what can and can't be done with process.nextTick.
So I have an array of a million elements that I'm sorting in Node.JS. I want Node to be responsive to other requests while it's doing this.
Is there any way to make Array.prototype.sort() not block other processes? Since this is a core function, I can't insert any process.nextTick().
I could implement quicksort manually, but I can't see how you do that efficiently, in a continuation-passing-style, which seems to be required for process.nextTick(). I can modify a for loop to do this, but sort() seems impossible.
While it's not possible to make Array.prototype.sort somehow behave asynchronously itself, asynchronous sorting is definitely possible, as shown by this sorting demo to demonstrate the speed advantage of setImmediate (shim) over setTimeout.
The source code does not seem to come with any license, unfortunately. The Github repo for the demo at https://github.com/jphpsf/setImmediate-shim-demo names Jason Weber as the author. You may want to ask him if you want to use (parts) of the code.
I think that if you use setImmediate (available since Node 0.10) the individual sort operations will be effectively interleaved with I/O callbacks. For such a big amount of work, I would not recommend process.nextTick (if it works at all, because there's a 1000 maxTickDepth limit). See setImmediate vs. nextTick for some backgroud.
Using setImmediate instead of plain "synchronous" processing will certainly be slower overall, so you could choose to handle a batch of individual sort operations per "tick" to speed things up, at the expense of Node not being responsive during that time. I think the right balance between speed and responsiveness wrt I/O can only be found with experimentation.
A much simpler alternative would be to do it more like web workers: Spawn a child process and do the sorting there. Biggest problem you face then is transferring the sorted data back to your main process(to generate some kind of output, presumably). AFAIK there's nothing like transferable objects for Node.js. After having buffered the sorted array, you could stream the results to the child process stdout and parse the data in the main process, or perhaps more simple; use child process messaging.
You may not have a spare cpu core lying around, so the child process would invade some other process cpu time. To avoid the sort process from hurting your other processes, you may need to assign it a low priority. It's seemingly not possible to do this with Node itself, but you could try using nice, as discussed here: https://groups.google.com/forum/#!topic/nodejs/9O-2gLJzmcQ . I have no experience in this matter.
Well, I initially thought you could use async.sortBy, but upon closer examination it seems that won't behave as you need. See Array.sort and Node.js for a similar question, although at the moment there's no accepted answer.