Socket.io connecting to server but not establishing connecting with any service

So, I am trying to set up a socket.io thing for relaying chat and other messages back and forth on a page in my site, but I am having trouble getting it up and running. What happens is, I start my server, and open the client page and the server sees that someone is trying to connect and says a bunch of stuff. From the client's point of view it tries to connect with websockets, that times out, then xhr-polling, which also times out, then jsonp-polling, which also times out, then the connection fails. Sometimes after a while it connects for a second, then drops it, but that is not ideal. I'm trying to connect with chrome, and had set up another websocket server that worked, so I know websockets is at least possible, but not cross browser compatible enough for my needs, hence the socket.io. I would like to know what I am doing wrong with getting it to connect properly.

server.js

var io = require('socket.io').listen(8080);

io.sockets.on('connection', function (socket) {
    socket.emit('connect_event', { hello: 'world' });
    socket.on('message', function (msg) { socket.emit('message', msg);});
    socket.on('disconnect', function () { });
});

client.php

<script src="http://domain.com:8080/socket.io/socket.io.js"></script>
<script>
var socket = io.connect('domain.com',{port: 8080});
socket.on('connecting', function (e) {console.log("attempting to connect with: "+e)});
socket.on('disconnect', function (e) {console.log(e)});
socket.on('connect_failed', function () {console.log("Connect Failed")});
socket.on('error', function (e) {console.log(e)});
socket.on('message', function (e) {console.log(e)});
socket.on('reconnect_failed', function (e) {console.log(e)});
socket.on('reconnect', function (e) {console.log(e)});
socket.on('reconnecting', function (e) {console.log(e)});

socket.on('connect', function () {
    socket.send('hi');
    console.log('connected');
});
socket.on('message', function (msg) {
    console.log(msg);
});
</script>

JS console output in browser:

attempting to connect with: websocket client.php:5
attempting to connect with: xhr-polling client.php:5
attempting to connect with: jsonp-polling client.php:5
Connect Failed client.php:7

server debug info:

debug - served static content /socket.io.js
debug - client authorized
info  - handshake authorized yJKfSwjarsZxert_Ij5V
debug - setting request GET /socket.io/1/websocket/yJKfSwjarsZxert_Ij5V
debug - set heartbeat interval for client yJKfSwjarsZxert_Ij5V
debug - client authorized for
debug - websocket writing 1::
debug - websocket writing 5:::{"name":"connect_event","args":[{"hello":"world"}]}
debug - setting request GET /socket.io/1/xhr-polling/yJKfSwjarsZxert_Ij5V?t=1368452739259
debug - setting poll timeout
debug - discarding transport
debug - cleared heartbeat interval for client yJKfSwjarsZxert_Ij5V
debug - setting request GET /socket.io/1/jsonp-polling/yJKfSwjarsZxert_Ij5V?t=1368452749248&i=0
debug - setting poll timeout
debug - discarding transport
debug - clearing poll timeout

when the server is running I can go to domain.com:8080 and it says

Welcome to socket.io.

So the port is open also in netstat -tan it has this for confirming purposes:

tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN

Nothing is being added to the error.log, only the get request for the page is being added to the access log.

Socket.io version 0.9.14

node.js v 0.10.5 It was suggested to try different node versions (specifically 0.8.x), but they all seem to act the same.

Server is a VPS with Ubuntu 12.04 LTS