Node.JS Transform Streams - Data loss

Question

Node.JS Transform Streams - Data loss

I've got some stream problems with my code that I don't know how to fix.

Code excerpt: http://gist.github.com/5689522

Essentially, I've got a couple of transform streams inside a parent transform stream, but i've got problems passing along the data because I'm using push on the first stream (S0) which is not being transformed, and therefore just pusing data directly to the second stream (S1), and causing errors. Normally I would use .pipe() to connect streams, but I can't see how to do that from inside the transform stream as I want to pipe the input, not the output, and the _transform function only gives a single chunk (buffer) as an argument.

Any ideas how to do something like this?

node.js
stream
buffer

Answer 1

you need to collect chunks together

var data='', tstream = new stream.Transform();
tstream._transform = function (chunk, encoding, done) {
    data += chunk.toString();
    done();
};

then reassign _flush function:

tstream._flush = function(done){
    data += 'hola muheres';
    this.push(data);
    done();
};

so all together:

req.pipe(anotherstream).pipe(tstream).pipe(response);

=> "somedata" => "somedatahola muheres"

Answer 2

From the documentation for push:

Note: This function should be called by Readable implementors, NOT by consumers of Readable streams.

Since you are not calling that in the implementation of ParserStream, you should not be calling _s0Stream.push, you should be doing _s0Stream.write. You probably want to also pass your done callback in that case too.