"Psycopg2" in Python 2.x blocks GIL

Question

I am trying to resolve a dispute with a colleague. Say I have a Python 2.6 application that uses psycopg2 to communicate with a Postgres database. The application is multithreaded. When a thread calls a database call using psycopg2 , does it release the GIL so that other threads can also call the database?

Answer 1

With a quick look at the Psycopg release notes , there are many links to the GIL release. Therefore, apparently, he is trying to release the GIL when appropriate. You did not ask about a specific scenario, so I do not think that a more specific answer is possible.

Answer 2

As you said, there is a rule that only the thread that acquired the GIL can work with Python objects or call Python / C API functions. To emulate concurrency execution, the Python interpreter regularly tries to switch threads. The lock is also released around potentially blocking I / O operations, such as reading or writing a file, so that other Python threads can work at the same time.

Most extension code (including psycopg2 code) that controls GIL has the following simple structure:

 Save the thread state in a local variable. Release the global interpreter lock. ... Do some blocking I/O operation ... Reacquire the global interpreter lock. Restore the thread state from the local variable. 

This means that when a blocking I / O operation (waiting for a network response from Postgres, for example) occurs, the GIL is freed, and other threads can continue to execute. When the I / O blocking operation is completed, the thread tries to get the GIL and continues execution (processing the results, etc.), when it finally acquires it.

Take a look at the implementation of psycopg2 here .