Nodejs/express, shutdown gracefully

I have an nodejs server running express to server out http calls. Is there a recommended approach to shutting down the server gracefully if something bad happens? Should I leave the server running somehow?

I.E. on an uncaught exception the server just stops, I think this will kill over connected clients and not hand them back a response.

Should I:

  1. Wait for all http connections to complete before I allow the server to die (and then restart it).
  2. Or should I try and stop the server from ever dieing?

is this proper?

process.on('exit', function () {
  console.log('About to exit, waiting for remaining connections to complete');
  app.close();
});

In this case an error might have been thrown leaving the server in an undefined state and the server will continue to finish up remaining connections.

Is there any good way to handle errors keep running or should I let the server die and restart?

Don't try to do anything fancy with unhandled exceptions. Let the server die.

Generally, if a route handler throws an exception, Express will simply catch it and return a HTTP 500 to the client. Since Express caught the exception, your server will not crash.

What usually happens that brings down a server is when an exception gets thrown inside a callback; for example:

app.get('/foo', function(req, res) {
    db.query(..., function(err, r) {
        throw new Error(); // oops, we'll crash
    });
});

Because the callback naturally executes outside the Express router call stack, there's no way to associate it with a specific request. However, you can guard against such a situation:

app.get('/foo', function(req, res, next) {
    db.query(..., function(err, r) {
        try {
            throw new Error();
        } catch(ex) {
            next(ex);
        }
    });
});

Express' next function takes an error as its first argument. If you call next with an error argument, it will stop processing routes and look for an error handler, or just return a 500.

Of course, wrapping everything in try/catch is probably overkill; most things don't actually throw exceptions. You should only do this if you know something might throw. Otherwise, you might end up in a very difficult to debug inconsistent state by swallowing exceptions.

It's important to remember that once an unexpected exception is thrown, your application is in an undefined state. It's possible that the problem request will never complete, and your app would never restart. Or any number of other strange things could happen. (Was it a database issue? Filesystem? Corrupted memory?)

These kinds of situations are incredibly difficult to diagnose, much less debug. It's safer and arguably better to just fail fast, crash, and restart the server quickly. Yes, any clients that happen to be connected would get cut off, but it's easy enough for them to simply retry.

If you're worried about availability, use the cluster module so that you have several (usually number of CPU cores) server processes running, and one crashing won't bring the entire site down.

NodeJS will shutdown when it's not expecting anything to happen; While the http server is waiting on connections node stays running.

server.close([callback])

Stops the server from accepting new connections and keeps existing connections. This function is asynchronous, the server is finally closed when all connections are ended and the server emits a close event. Optionally, you can pass a callback to listen for the close event.

After you do this the server will continue running until the last connection has finished and then shutdown gracefully.

If you don't want to wait for connections to terminate you can use socket.end() or socket.destroy() on your remaining connections.

See http://nodejs.org/api/net.html


process.on('exit', function () {
    console.log('About to exit, waiting for remaining connections to complete');
    app.close();
});

This function should run when node exists and not actually tell node to exit. Here is one way you can end your process but it wouldn't end gracefully:

process.exit([code])

Ends the process with the specified code. If omitted, exit uses the 'success' code 0.

To exit with a 'failure' code:

process.exit(1); The shell that executed node should see the exit code as 1.

See http://nodejs.org/api/process.html


I hope this helps!