MongoDB findAndModify from multiple clients

Question

my MongoDB collection is used as a job queue, and there are 3 C ++ machines that are read from this collection. The problem is that these three cannot do the same job. All tasks need to be done only once.

I retrieve all the incomplete jobs by looking at the collection for all records using "isDone: False", and then update this document "isDone: True". But if 2 cars find the same document at the same time, they will work for the same job. How can i avoid this?

Edit: My question is - does findAndModify really solve this problem? (After reading How to provide Exclusive Reading in FindOndModify MongoDb? )

Answer 1

Yes, findAndModify will solve it.

Reference: MongoDB findAndModify from several clients "... Note: This command obtains a write lock on the corresponding database and blocks other operations until it is completed; however, as a rule, write locks are short-lived and equivalent to other similar update () operations. ... "

Link: http://docs.mongodb.org/manual/reference/method/db.collection.update/#db.collection.update "... For insecure collections, you can override this behavior with an isolated isolation operator that isolates update operation and blocks other write operations during update. See Isolation statement. ... "

Link: http://docs.mongodb.org/manual/reference/operator/isolated/

Regards, Moacy

Answer 2

Yes, find-and-modify will solve your problem:

db.collection.findAndModify ({query: {isDone: false}, update: {$ set: {isDone: true}}, new: true, upsert: false # never create new documents});

This will return the one document that he just updated from false to true.

But you have a serious problem if your C ++ clients have ever had hiccups (the box dies, they are killed, the code has an error, etc.). Imagine that your TCP connection drops right after the update on the server, but before the C ++ code receives the job. It is usually better to have a multiphase approach:

• change "isDone" to "isInProgress", then when this is done, delete the document. (Now you can see the stack of “todo” and “done.” If something is “done” for a long time, the client probably died.

• change "isDone" to "phase" and atomically set it from "new" to "initial" (and then set to "finished"). Now you can see that something has been “running” for a long time, perhaps the client has died.

If you are really complex, you can make a partial index. For example, “Only index documents with“ phase: {$ ne: “completed”}. ”Now you do not need to waste space indexing millions of finished documents. The index contains only a few new / incomplete documents, so it is smaller / faster.