I'm looking at implementing a node server which will be receiving uploads of potentially large files and forwarding the data on through another stream. I've found this article:
http://www.componentix.com/blog/13/file-uploads-using-nodejs-once-again
Which has some useful code examples around handling the various events as well as the pump problem with different speeds of the streams on both sides. What's still not clear to me (and what I can't seem to find documentation for) is when exactly the 'data' event is emitted for the incoming stream by node.
The node docs state:
Event: 'data'
Emitted when data is received. The argument data will be a Buffer or String. Encoding of data is set by socket.setEncoding(). (See the Readable Stream section for more information.)
What is meant by "when data is received"? Is this fired when the incoming data chunk reaches a certain size? When the incoming connection is closed? After a certain time?
The stream has an internal buffer that it uses to store the data until it's ready to fire the data event. That might be a few cases depending on the type of stream: internal buffer full, all data read, connection closed, etc.
The network stream is probably firing the data event with whatever data received from the socket's read method. If I can find it in the node source, I'll reference it.