Geek Answers Handbook

Python Threads in C

I am writing a multi-threaded program in C. Before creating the threads, the global python environment is initialized by calling Py_Initialize() . Then, in each created thread, the global python environment is shared, and each thread calls the python method with parameters converted to C. Everything works well before that.

When I use time.sleep() in loaded python modules, the C program raises the value of Segmentation Fault . In addition, the loaded python module must load another C lib to continue. I wrote the following dumb lib counter to check it:

 # python part, call the counter function lib = ctypes.cdll.LoadLibrary(libpycount.so) for i in xrange(10): lib.count() 
 // C part, dummy countings #include <stdio.h> int counter = 1; void count() { printf("counter:%d \n", counter); counter++; }

I guess this could be because I did not cope with the complex creation of threads in the right direction. And I found Non-Python created threads in a python document.

Any ideas or suggestions?

+7
source share
2 answers

My problem is resolved. You may have your problems more specific, so I'm trying to write my solution in a more general way. Hope this helps.


- In main stream C

  • initialize the Python environment at the very beginning:
 /*define a global variable to store the main python thread state*/ PyThreadState * mainThreadState = NULL; if(!Py_IsInitialized()) Py_Initialize(); mainThreadState = = PyThreadState_Get();
  • Then run the C threads:
 pthread_create(pthread_id, NULL, thread_entrance, NULL);



- In each thread, or we can say in the body of the thread_entrance function

  • prepare Wednesday:
 /*get the lock and create new python thread state*/ PyEval_AcquireLock(); PyInterpreterState * mainInterpreterState = mainThreadState->interp; PyThreadState * myThreadState = PyThreadState_New(mainInterpreterState); PyEval_ReleaseLock(); /*don't forget to release the lock*/ /* * some C manipulations here */
  • enter the python code here:
 /*get the lock and put your C-Python code here*/ PyEval_AcquireLock(); PyThreadState_Swap(myThreadState); /*swap your python thread state*/ PyEval_CallObject(py_function, py_arguments); /*or just something like PyRun_SimpleString("print \"hello world\""); for test*/ PyThreadState_Swap(NULL); /*clean the thread state before leaving*/ PyEval_ReleaseLock();



- return to main stream C

  • when each thread finishes its work, python environment terminates
 pthread_join(pthread_id, NULL); PyEval_RestoreThread(mainThreadState); Py_Finalize();
+4
source

The question is whether the Python interpreter is thread safe. This is what the documentation says about running multiple interpreters in the same process space;

Errors and caveats: since sub-interpreters (and the main interpreter) are part of the same process, isolation between them is not perfect - for example, using low-level file operations like os.close () they can (accidentally or maliciously) influence open each other files. Because extensions are distributed between (sub) interpreters, some extensions may not work correctly; this is especially likely when the extension uses (static) global variables or when the extension manages its modular dictionary after it is initialized. You can insert objects created in one sub-interpreter into the namespace of another sub-interpreter; this should be done with extreme caution in order to avoid sharing between the user functions, methods, instances, or classes between sub-interpreters, since the import operations performed by such objects can affect the incorrect (sub-) interpreter of loaded modules. (XXX) It's hard to fix a bug that will be addressed in a future version.)

... and I don't think Python threads are the same as native threads, such as those found in C / C ++

+2
source

More articles:


All Articles