Do I need a lock when using multi-threaded python?

Question

Think about this code:

#!/usr/bin/env python from threading import Thread count = 0 def test(): global count for i in range(10): count = count + 1 if __name__ == '__main__': for i in range(1000): Thread(target = test).start() print count 

I use multiple threads, but the result is always correct. Does this mean that I can use python threads without blocking when implementing something like a visitor counter?

Answer 1

You need it. Although multithreading works differently in Python, due to Global Interpreter Lock, operations that are not atomic in Python bytecode still require locking.

In this case, you can check the bytecode for your test ( dis.dis(test) ) function:

  3 0 SETUP_LOOP 30 (to 33) 3 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (1000) 9 CALL_FUNCTION 1 12 GET_ITER >> 13 FOR_ITER 16 (to 32) 16 STORE_FAST 0 (i) 4 19 LOAD_GLOBAL 1 (count) # start of increment 22 LOAD_CONST 2 (1) 25 BINARY_ADD 26 STORE_GLOBAL 1 (count) # end of increment 29 JUMP_ABSOLUTE 13 >> 32 POP_BLOCK >> 33 LOAD_CONST 0 (None) 36 RETURN_VALUE 

As you can see, the increment is 2xload, update, store at the bytecode level, so this will not work. An increment is actually 4 separate operations that you must protect to ensure that they are not interrupted.

In your example, the problem remains, even if you use count += 1 , as the bytecode shows:

 4 19 LOAD_GLOBAL 1 (count) 22 LOAD_CONST 2 (1) 25 INPLACE_ADD 26 STORE_GLOBAL 1 (count) 

Answer 2

You will not need a lock if you just assigned.

But how do you do count = count + 1 , something can happen between every reading of count , adding 1 and writing to count .

Even using count += 1 will not solve this problem, since it is also related to assignment. (Since inplace operations are associated with an appointment under the hood, the situation is the same.)

Answer 3

You should definitely use Lock. You have the correct answer in such a simple case. Try to make it for the range (100000) in the main . You will see the problem. In my car, the result is 999960, but this is a random result. Errors will occur depending on the system load, etc.