I am seeking a way to serialize tasks that affect data in a data store without using MySQL ex: A worker doing accounting on group1 should be the only worker doing accounting on group one and should wait in queue if another worker is doing accounting on group one.
I could accomplish this with MySQL by setting up semaphore table, start a transaction, do a update on the the the row for group1, do my task, and commit.
i was thinking that maybe 0mq redis or some sort of messaging system could be used accomplish the same goal and allow me to use what ever data store i want.
i was also thinking that ScalienDB may be able to solve the problem in the same manor as mysql seeing that it supports transactions. The documentation for ScalienDB seems to be somewhat incomplete so i can't quite ascertain if it can do transactions in that fashion.
So my questions are:
1. can ScalienDB do a transaction that would force a client to wait for another client to commit if it wanted to edit a row in a table that another client has also done a edit on.
2. using a messaging system how would you suggest implementing something that boils down to something like this:
var semaphore = semaphore_group()
semaphore.acquire('task1',function(){
// do work after a sophomore is locked in
semaphore.release() //
})
ideally i would not want this system to dependent on a centralized broker
3. is there a alternate solution that would fit this problem
The thing I love most about Node is no-threads which means every line of code can be considered atomic. So, you can easily write just a few lines of JS code which will run a socket server that will provide workers the exact mechanism you want to have as shown in your example code.
I am not sure if it is technically right to call this Semaphore since we keep track of workers waiting for their turns.
Workers:
var Semaphore = reqire('./semaphore.js'),
semaphore = new Semaphore(worker-id, group-id);
semaphore.on('ready', function() {
// Yay! We can work..!
semaphore.done();
}
semaphore.js skeleton:
// imports...
module.exports = S = function(gid) {
events.EventEmitter.call(this);
// create socket connection to semaphore server
// send a message for lock on gid
// emit ready event when server sends ready message.
// send 'done' message when done() method is called.
}; util.inherits(S, events.EventEmitter);
semaphore-server.js skeleton:
// imports...
var queues = {}; // { 'group-id': [worker1, worker2, ...] }
// start socket server
/* on message: take worker-id and group-id
queues[groupid].push(workerid);
check if queues[groupid][0]==workerid and if it is,
send back ready message.
*/
/* on message 'done', remove workerid from queues[groupid]
and if there are any workers waiting, send the first one
ready message.
*/
Your code will not be much longer than just this skeleton above. ZeroMQ or any other messaging is not necessary for such simple messaging system. You might consider adding timeouts to workers and whenever timeout occurs, send a message to worker asking "are you still working?" and continue according to that. Your timeout handler can also check if there is any other worker on the queue so if there is not, it is not even necessary to bother the worker for time...
I generally prefer using the tools I already have in hand instead of going to third parties if all I need is just a simple system I can build in an hour like this. Any other library/dbms/messaing-system or whatever will add complexity and administration drudgery to my software which I do not like. Since you are already building something in Node and it already perfectly provides what you want, I believe this is the way to go.
I would consider using a distributed coordination service like Apache Zookeeper to create and handle your distributed locks. Here is a recipe describing how you would go about it: http://zookeeper.apache.org/doc/trunk/recipes.html#sc_recipes_Locks