I am trying to build a server setup with the following:
And I am trying to figure out a good way to help them all communicate with each other, as well as clients such as a browser, or mobile. The browser part will also be hosted on the Server N with the node apps. I want to do this in a scalable way so I can simply add a new node server and have it join the others and simply be able to start communicating with them or to be available to process jobs.
I had been planning on using sockets originally. But after reading into json-rpc (https://github.com/uber/multitransport-jsonrpc) it seems like it could be a better option for everything to talk to each other in a clean, and stateless manner. The reason for wanting to do this, is to try and help scalability. Which is why rpc made sense to me over sockets.
In researching this, I also saw another SO answer mention that using Redis for this 'message passing' between servers works well too. I was planning on using Redis for sessions anyways. But im not sure if I should go that route since I am not just message passing between servers, but from clients/mobile as well (FYI I am familiar with the message passing concepts, including from erlang as well). Using Redis also doesn't seem like it would be as fast as sockets or rcp for this kind of communication. I could be wrong.
Another reason I want to do it like this, is because if I ever have to replace a component for some reason, I can do that. Such as if I have say 10 node servers and want to test some new code to do X, I can deploy it to just one server and still have the same input/output and test the performance on it. Like the linux philosophy of making smaller apps that do one thing.
So I was curious about some good ways to go about this. Should I use rpc for this? Or sockets? Since I want to use redis anyways for session management and some other things down the line, I will eventually have that installed anyways. Should/Would I use both sockets and rpc together? Or am I even going about this with the wrong approach.