In node.js programming, how do you handle the scenario where a callback doesn't fire?

Question

In node.js programming, how do you handle the scenario where a callback doesn't fire?

A frequently used asynchronous function call idiom in node.js is to use callback functions like this:

library.doSomething(params, function(err,result) {
    if (err) {
        ... handle error, retry etc
    } else {
        ... process results, be happy!
    }
});

It's great - you call something, then later on handle the results or an error. There is unfortunately an excluded third option... that the code you've called never executes your callback. What are the best approaches to handle the possibility of a callback never getting called?

A lot of the time, especially if you're writing a library that relies on the network, you need to guarantee that you will call any callbacks passed to you once and only once. With that in mind, a pattern something like this looks like the way to go:

// set a timout
var failed = false, callbackFailure = setTimeout(function() {
    ... handle failure, call further pending callbacks with a timeout error
    failed = true;
},30000);

library.doSomething(params, function(err,result) {
    if (!failed) {
        clearTimeout(callbackFailure);
        if (err) {
            ... handle error, retry etc
        } else {
            ... process results, be happy again!
        }
    }
});

It seems like a matter of faith that any callback you expect to fire actually will fire, and I'm sure all programmers have run into scenarios where callbacks simply won't execute for whatever reason - cosmic rays, sunspots, network failures, bugs in a third party library, or ... gasp ... bugs in your own code.

Is something like my code example actually a good practice to adopt or has the node.js community already found a better way to handle it?

node.js

Answer 1

I'm not sure where it's gone, but an answer popped in here and disappeared with a link to https://github.com/andyet/paddle - a small library designed to provide callback execution "insurance". At least it suggests I'm not the first person to have scratched my head regarding this problem.

From the docs on there is an anecdote validating the question somewhat:

Node.js famously had an http client but where occasionally no callback would occur for an HTTP request if the response was too fast. If I wanted to make sure my http client callback occurred.

The example they give is a little more sophisticated than my example and can handle event based callbacks, enabling code to regularly "check in" as intermediate activity like an ondata handler fires and then trigger an error if it stops or times out.

setTimeout(function() {
    paddle.stop();
}, 12000);

var req = http.get(options, function(res) {
    var http_insurance = paddle.insure(function(res) {
        console.log("The request never had body events!");
        console.log('STATUS: ' + res.statusCode);
        console.log('HEADERS: ' + JSON.stringify(res.headers));
    }, 9, [res]);
    res.setEncoding('utf8');
    res.on('data', function (chunk) {
        console.log('BODY: ' + chunk);
        http_insurance.check_in();
    });
});
req.end();

I'm answering my own question here, but I would still be interested to see if any other implementations, libraries or patterns exist to solve the same problem.

Answer 2

Not looking at paddle it seems easy enough to create callback execution insurance for badly written or debugged code.

A function like this:

// requireCB:
//   timeout - milliseconds before insurance gets used
//   cb - callback function to invoke normally or when insurance expires
function requireCB(timeout, cb) {
    var timeoutTimer;
    var canBeCalled = false;
    var myCB = function () {
        if (timeoutTimer) { clearTimeout(timeoutTimer); timeoutTimer = 0; }
        if (canBeCalled) { canBeCalled = false; cb.apply(this, arguments); }
    };
    timeoutTimer = setTimeout(function () { myCB(new Error('timed out')) }, timeout);
    canBeCalled = true;
    return myCB;
}

We create insurance widgets that can be used like so:

var rcb = requireCB(100, function (err, data1, data2) {
    console.log('called, err =', err, ', data1 =', data1, ', data2 =', data2);
});
functionExpectedToInvokeCallback(rcb);

Or in an actually immediately runnable example:

var rcb = requireCB(100, function (err, data) {  // set insurance for 0.1 seconds
    console.log('called, err =', err, ', data =', data);
});
setTimeout(function () { rcb(undefined, 'sheep') }, 1000);  // set normal callback in 1 second

Flip the timeout values to not trigger the insurance.