Concurrency using Sidekiq causes some problems

Question

I use Sidekiq in my rails application to create 50,000+ jobs at the same time. The size of our pool is 9.

The works are related to each other and do the same. We have another model on which there is a counter. During each task, we check whether this model has a column with a value above 200. If it is above 200, we create another instance of this model with a value = 0 and continue working. However, since we have 9 tasks to be performed at a time, all 9 tasks read the value of this column, which should be more than 200 at the same time, and everyone creates new instances, which is incorrect.

What is the best way to solve this problem? Basically, we want all assignments to be read from the most recent value.

Answer 1

I cannot post any specific code, because it will depend heavily on the type and settings of your database, but you should try locking the database.

When reading a table, an employee must block it until it finishes creating a new record with the value 0. You must lock the table for reading so that other employees have to wait until this worker finishes. It is also possible to lock individual lines, but I do not know if this will work in your case.

Answer 2

Suppose your model is called Counter .

First find / create the appropriate counter:

counter = Counter.where('count < 200').first_or_create ...

(Note: this is not atomic! If you cannot have more than one active counter, then see:
Race conditions in Rails first_or_create and
How to avoid race conditions in a Rails app?

Then try to increase it atomically in the database:

 success = Counter.where(id: counter.id) .where('count < 200') .update_all('count = count + 1') 

If this worked, success == 1 otherwise success == 0 . If this works, use a counter, otherwise try again.

Answer 3

Redis is better suited for this type of operation, and you already have easy access to it through the Sidekiq Redis connection.

 value = Sidekiq.redis { |c| c.incr("my-counter") } if value % 200 == 0 # create new instance end 

Perhaps something like this will work for you.