Geek Answers Handbook

Python-C api concurrency issue

We are developing a small server application. The server application does some data processing and responds to the client. So that part of the data processing was configurable and flexible, we decided to go for scripting and based on the availability of various ready-made modules, we decided to go for Python. We use the Python-C api to send / receive data between c and python.

The algorithm works something like this: -

  • The server receives some data from the client, this data is stored in a dictionary created in c. The dictionary is created using the api function PyDict_New (); from c. The input is stored as a pair of key values ​​in the dictionary using the api function PyDict_SetItemString ();
  • Then we execute the python script PyRun_SimpleString (); passing the script as a parameter. This script uses a dictionary created in c. Note that we are creating a dictionary created in c that is accessible to the script using the PyImport_AddModule () methods; and PyModule_AddObject ();
  • We save the data processing result in a script as a pair of key values ​​in the same dictionary created above. Then the c code can simply access the result variable (key-value pairs) after the script is executed.

Problem The problem we are facing is related to simultaneous requests from different clients. When multiple requests come from different clients, we tend to exclude link reference number exclusions. Please note that for each request that comes to the user, we create an independent dictionary for this user. To overcome this problem, we included the call to PyRun_SimpleString (); in PyEval_AcquireLock (); and PyEval_ReleaseLock ();, but this led to the script being a blocking call. Therefore, if a script takes a long time to execute, all other users are also waiting for a response.

Could you suggest the best possible approach or indicate pointers to where we are making mistakes. Please call me for more information.

/ .

+5
3

, , .

+1

, http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock .

GIL, Python. PyRun_SimpleString GIL X-. .

Edit:

, , Python :

// acquire the lock and switch thread state
PyEval_AcquireLock();
PyThreadState_Swap(perThreadState);

// execute some python code
PyEval_SimpleString("print 123");

// clear the thread state and release the lock
PyThreadState_Swap(NULL);
PyEval_ReleaseLock();
+1

I suggest you study the module multiprocessing.

+1
source

More articles:


All Articles