Sending messages too quickly in nodejs 'net'

Question

Sending messages too quickly in nodejs 'net'

I'm sending json messages between 2 nodejs servers use the 'net' module.

It all works great, however if I try to send 2 messages too close to each other I get the error:

undefined:1
{"message":"wheelSpeed","data":{"left":0,"right":0}}{"message":"newPlayer","da
                                                    ^
SyntaxError: Unexpected token {
    at Object.parse (native)
    at Socket.<anonymous> (/home/pi/halloween/pi-node.js:128:32)
    at Socket.emit (events.js:95:17)
    at Socket.<anonymous> (_stream_readable.js:764:14)
    at Socket.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:426:10)
    at emitReadable (_stream_readable.js:422:5)
    at readableAddChunk (_stream_readable.js:165:9)
    at Socket.Readable.push (_stream_readable.js:127:10)
    at TCP.onread (net.js:528:21)

The code where this fails is:

var messageData = JSON.parse(rawData);

and the surrounding code looks like:

server.on('data', function(rawData) {

    console.log('From server', rawData);
    var messageData = JSON.parse(rawData);
    console.log('Remote server is talking to us', messageData);
    var data = messageData.data;

The output from that debugging before the crash is:

From server <Buffer 7b 22 6d 65 73 73 61 67 65 22 3a 22 6d 6f 76 65 4c 61 75 6e 63 68 65 72 22 2c 22 64 61 74 61 22 3a 7b 22 78 22 3a 39 30 2c 22 79 22 3a 39 30 7d 7d>
Remote server is talking to us { message: 'moveLauncher', data: { x: 90, y: 90 } }
From server <Buffer 7b 22 6d 65 73 73 61 67 65 22 3a 22 77 68 65 65 6c 53 70 65 65 64 22 2c 22 64 61 74 61 22 3a 7b 22 6c 65 66 74 22 3a 30 2c 22 72 69 67 68 74 22 3a 30 7d ...>

undefined:1
{"message":"wheelSpeed","data":{"left":0,"right":0}}{"message":"newPlayer","da

It looks like one message gets through OK but the next 2 fail as it seems they come in at the same time.

Is there a way to indicate the end of one message and the start of the next? If I leave a reasonable amount of time between messages this seems to happen automatically for me.

javascript
json
node.js

Answer 1

You are using TCP protocol, TCP protocol is used to send streams of bytes, so there is no concept of "message" or something like that implemented on the protocol itself

For example, this:

socket.write("ABC");
socket.write("DEF");

Could arrive as:

"ABC" and later "DEF"
"ABCDEF",
"AB" and later "CDEF"
Etc...

Any combo you can imagine, you cannot predict that

You have two big options on this:

Implement your own protocol on top of TCP to suit your needs
Switch to another protocol

Implement your own protocol on top of TCP

You need to implement your own protocol on top of TCP custom for your needs, according your question it seems like you need some kind of "message" separation, for example, you can establish on your protocol '\n' is the separator for messages. You WILL NEED to implement the logic to concat fragmented messages until you found a '\n' and a spliting logic for packets with multiple messages (because you can't control/predict the fragmentation on TCP). This is what telnet protocol does (telnet and raw TCP IS NOT exactly the same, because of what I just explained)

Switch to another protocol

This is the another option I personally prefer on cases like this, use another protocol (prefereably over HTTP), socket.io is my favourite one (http://socket.io/), any protocol is good for solve this issue this issue as long as it haves client/server libraries for node and it suits all your needs