Are Python extensions expanding from Cython / Pyrex?

Question

If not, is there a way to guarantee thread safety by programming in a certain way?

To clarify when it comes to threadafe, I am referring to Python threads, not OS-level threads.

Answer 1

It all depends on the interaction between your Cython code and the Python GIL, as described in detail here . Unless you do anything special, Cython-generated code will respect the GIL (like a C-encoded extension that does not use GIL-free macros); what makes such code “as thread safe as Python code” is a bit, but easier to process than completely free code for streaming (you still need to archive multi-threaded communication and synchronization, ideally with queue instances, but possibly with blocking and c).

Code that has abandoned the GIL and has not yet received it back SHOULD NOT in any way interact with the Python runtime and objects that the Python runtime uses — this is the same for Cython as it is for C-encoded extensions. The surface of this, of course, such code can run on a separate kernel (as long as it does not need to synchronize or in any way communicate with the Python runtime, of course).

Answer 2

Locking the global Python interpreter means that only one thread can be active in the interpreter at any given time. However, as soon as control is transferred to extension C, another thread may be active in the interpreter. You can create multiple threads, and nothing prevents the thread from being interrupted in the middle of a critical section. N

in thread-safe code it can be implemented in the interpreter, so nothing about code running inside the interpreter is essentially thread-safe. In C or Pyrex modules, code can modify the data structures that are visible to python code. Natural code can, of course, also have problems with threads with their own data structures.

You cannot guarantee the safety of the stream outside of the proper construction and synchronization - the GIL on the python interpreter will not significantly change this.