How to extract .tar.bz2 in node.js using streams?

Question

How to extract .tar.bz2 in node.js using streams?

I'm trying to extract some .tar.bz2 files in node.js. I'm searching here, in npm, github and teh google for this but there is no ready solution.

My files are ~25mb each so I think the best way would be in a piped stream with the tar module (similar to how you use the Gunzip from node.js's built-in ZLib library for .tar.gz). This way I can also extract directly from piped http using request.

I found https://github.com/Woodya/node-gzbz2 (and it's many renamed forks like gzbz) but they require external dependencies build using node-gyp. I don't want to use these since the module I'm building has to work without hassle on linux, mac and windows using just npm and without depending on external libraries like python.

Alternately I look at https://github.com/cscott/seek-bzip (and it's sources) and I like how it is pure javascript but it only decodes Buffers.

Can anybody advice me on the way to go here?

edit: The author of seek-bzip kindly created a wrapper to turn his synchronous streams into asynchronous ones, but this fix depends on node-fibers which again uses node-gyp which in my case is undesirable. See https://github.com/cscott/seek-bzip/issues/1

edit2: I'm still looking for a cross-platform solution, but here is a quick way to do this using CLI commands:

var cmd = 'bunzip2 -c ' + sourceFile + ' | (cd ' + targetDir + '; tar -xf -)';

require('child_process').exec(cmd, function (err, stdout, stderr) {
    if (err) {
        // bad
    }
    // yea!
});

javascript
node.js
stream
tar
bzip2

Answer 1

I feel like this question is really 2 questions: how to decrypt bz2 and how to untar. I'll answer the untaring part. The tar-stream module is a pretty good one:

var tar = require('tar-stream')    

var extract = tar.extract();
extract.on('entry', function(header, stream, callback) {
    // make directories or files depending on the header here...
    // call callback() when you're done with this entry
});

fs.createReadStream("something.tar").pipe(extract)

extract.on('finish', function() {
    console.log('done!')
});