I've finally finished converting one of my projects to use Node.JS, but now I'm having problems keeping my app running on the server :/ The problem is that if I close my putty session Node just stops.
I've done a lot of searching on this problem, and it seems that creating an upstart script and using the Forever module is the way to go.
I started googling and created this upstart script:
#!upstart
description "Loner NodeJS app launcher"
author "me@me.com"
start on startup
stop on shutdown
script
export HOME="/root"
exec sudo node /home/jjmpsp/server.js >> /home/jjmpsp/server.sys.log 2>&1
end script
I then ran start app on the server last night and the server stayed running when I closed the putty session. All good.
However, I came on this morning and discovered that the Node app had stopped so I checked the server.sys.log file to see what was going on. It seems that the app ran fine until it eventually encountered this exception:
debug: client authorized
info: handshake authorized fziLHZA3Vo9i55eubvOq
events.js:48
throw arguments[1]; // Unhandled 'error' event
^
Error: Connection lost: The server closed the connection.
at Protocol.end (/home/jjmpsp/node_modules/mysql/lib/protocol/Protocol.js:73:13)
at Socket.onend (stream.js:80:10)
at Socket.emit (events.js:88:20)
at TCP.onread (net.js:348:51)
Today I've been googling even more and discovered that Forever will actually restart a NodeJS app if it exits unexpectedly. I tried installing the module with npm install forever but I get this huge list of errors:
jjmpsp@alex:~$ npm install forever
npm ERR! error installing forever@0.9.2 Error: No compatible version found: node-fork@'>=0.4.0- <0.5.0-'
npm ERR! error installing forever@0.9.2 No valid targets found.
npm ERR! error installing forever@0.9.2 Perhaps not compatible with your version of node?
npm ERR! error installing forever@0.9.2 at installTargetsError (/usr/local/lib/node_modules/npm/lib/cache.js:424:10)
npm ERR! error installing forever@0.9.2 at /usr/local/lib/node_modules/npm/lib/cache.js:406:17
npm ERR! error installing forever@0.9.2 at saved (/usr/local/lib/node_modules/npm/lib/utils/npm-registry-client/get.js:136:7)
npm ERR! error installing forever@0.9.2 at Object.cb [as oncomplete] (/usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:36:9)
npm ERR! Error: No compatible version found: node-fork@'>=0.4.0- <0.5.0-'
npm ERR! No valid targets found.
npm ERR! Perhaps not compatible with your version of node?
npm ERR! at installTargetsError (/usr/local/lib/node_modules/npm/lib/cache.js:424:10)
npm ERR! at /usr/local/lib/node_modules/npm/lib/cache.js:406:17
npm ERR! at saved (/usr/local/lib/node_modules/npm/lib/utils/npm-registry-client/get.js:136:7)
npm ERR! at Object.cb [as oncomplete] (/usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:36:9)
npm ERR! Report this *entire* log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <npm-@googlegroups.com>
npm ERR!
npm ERR! System Linux 3.8.4-x86_64-linode31
npm ERR! command "node" "/usr/local/bin/npm" "install" "forever"
npm ERR! cwd /home/jjmpsp
npm ERR! node -v v0.5.11-pre
npm ERR! npm -v 1.0.106
npm ERR! Error: EACCESS, Permission denied 'npm-debug.log'
npm ERR! Report this *entire* log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <npm-@googlegroups.com>
npm ERR!
npm ERR! System Linux 3.8.4-x86_64-linode31
npm ERR! command "node" "/usr/local/bin/npm" "install" "forever"
npm ERR! cwd /home/jjmpsp
npm ERR! node -v v0.5.11-pre
npm ERR! npm -v 1.0.106
npm ERR! path npm-debug.log
npm ERR! code EACCESS
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/jjmpsp/npm-debug.log
npm not ok
What steps should I take to fix this? I'm completely out of ideas. I've been googling all sorts of technical details and I'm just not getting anywhere.
Any help is greatly appreciated :)
First, you should focus on what is killing your node server. Forever isn't going to "fix" the problem. Every time node exits/restarts it will cause problems and your users can loose data. upstart or forever are just band-aids. (In fact, Upstart will restart your server, but it will give up if your server doesn't stay running.)
The only long-term solution to find/fix each source of errors and write a regression test suite to verify that each previous problems has been fixed.
start on startup will not work
Your forever install broke because of permissions. Try sudo npm install -g forever
Advanced:
Your entire server setup should be scripted. That way you can set up a test/staging server that mirrors production. one simple way to do this is Vagrant. You can develop "locally" on the same OS that's on your server. And you can test things like "does node come up when I reboot?" or "if the server gets wiped out, can I re-create my server from the base OS?"
After a painful few hours of re configuring everything, I think I have finally solved my problem! It looks like I may have had 2 versions of node or something! For future reference: If you are new to Node, be sure to install nvm to make managing node versions easier and you won't experience this problem :)
I think forever doesn't exits your nodejs app unexpectedly, It might be because of the mysql 'wait_timeout' parameter which is set to 28800 second (i.e 8 hours) by default. If the nodejs app does not interact with mysql database for 8 hour, then the connection gets closed. You have to set the wait_timeout value to 'MAX value'(i.e 365 days) or any other value as you required in the mysql db settings.
Link for the wait_timeout settings:
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_wait_timeout
or
Add the following code in your code. This code will ping the mysql db once in an hour and your connection will not break.
// MYSQL ping code starts
function keepalive()
{
conn.query('select 1', [], function(err, result)
{
if(err)
return console.log(err);
// Successul keepalive
console.log("keepalive: "+ result);
});
}
setInterval(keepalive, 1000*60*60);
// MYSQL ping code ends
You can also use the handledisconnect() code and handle the particular connection error in the code as given in the example below.
///Code starts
function handledisconnect() {
log.info("handledisconnect()");
// Reading config.json file for database connection
var db_config = JSON.parse(fs.readFileSync(appRoot + "/conf.json"));
conn = mysql.createConnection(db_config);
conn.connect(function(err) {
if (err) {
log.error('error when connecting to db : ',err);
// We introduce a delay before attempting to reconnect
setTimeout(handledisconnect, 200);
} else {
log.info("database connected");
}
});
conn.on('error', function(err) {
log.error('db error : ',err);
// handling the reconnection for 'PROTOCOL_CONNECTION_LOST' error
if (err.code === 'PROTOCOL_CONNECTION_LOST') {
log.error(err);
setTimeout(handledisconnect, 200);
handledisconnect();
} else {
log.error(err);
}
// handling the reconnection for 'EADDRINUSE' error
if (err.code === 'EADDRINUSE') {
setTimeout(handledisconnect, 200);
handledisconnect();
} else {
log.error(err);
}
// handling the reconnection for 'ECONNREFUSED' error
if (err.code === 'ECONNREFUSED') {
log.error(err);
setTimeout(handledisconnect, 200);
handledisconnect();
} else {
log.error(err);
}
});
///**** ADD YOUR CODE HERE****////
}
handledisconnect();
/// Code ends
Use keymetrics.io, it will restart your node application automatically if any issue occurs.
http://unix.stackexchange.com/questions/479/keep-ssh-sessions-running-after-disconnection
do this:
screen // then press enter
node server.js
From the logs what actually seems to be the problem in your case is your mysql connection. Whenever the mysql server closes the connection the node server would die. You can use a keealive or use connection pooling.