Embedding Python in Multi-Threaded C/C++ Applications
Before your threaded C program is able to make use of the Python API, it must call some initialization routines. If the interpreter library is compiled with thread support enabled (as is usually the case), you have the runtime option of enabling threads or not. Do not enable runtime threading support unless you plan on using threads. If runtime support is not enabled, Python will be able to avoid the overhead associated with mutex locking its internal data structures. If you are using Python to extend a threaded application, you will need to enable thread support when you initialize the interpreter. I recommend initializing Python from within your main thread of execution, preferably during application startup, using the following two lines of code:
// initialize Python Py_Initialize(); // initialize thread support PyEval_InitThreads();
Both functions return void, so there are no error codes to check. You can now assume the Python interpreter is ready to execute Python code. Py_Initialize allocates global resources used by the interpreter library. Calling PyEval_InitThreads turns on the runtime thread support. This causes Python to enable its internal mutex lock mechanism, used to serialize access to critical sections of code within the interpreter. This function also has the side effect of locking the global interpreter lock. Once the function completes, you are responsible for releasing the lock. Before releasing the lock, however, you should grab a pointer to the current PyThreadState object. You will need this later in order to create new Python threads and to shut down the interpreter properly when you are finished using Python. Use the following bit of code to do this:
PyThreadState * mainThreadState = NULL; // save a pointer to the main PyThreadState object mainThreadState = PyThreadState_Get(); // release the lock PyEval_ReleaseLock();
Python requires a PyThreadState object for each thread that is executing Python code. The interpreter uses this object to manage a separate interpreter data space for each thread. In theory, this means that actions taken in one thread should not interfere with the state of another thread. For instance, if you throw an exception in one thread, the other snippets of Python code keep running as if nothing happened. You must help Python to manage per-thread data. To do this, manually create a PyThreadState object for each C thread that will execute Python code. In order to create a new PyThreadState object, you need a pre-existing PyInterpreterState object. The PyInterpreterState object holds information that is shared across all cooperating threads. When you initialized Python, it created a PyInterpreterState object and attached it to the main PyThreadState object. You can use this interpreter object to create a new PyThreadState for your own C thread. Here's some example code which does just that (ignore line wrapping):
// get the global lock PyEval_AcquireLock(); // get a reference to the PyInterpreterState PyInterpreterState * mainInterpreterState = mainThreadState->interp<\n>; // create a thread state object for this thread PyThreadState * myThreadState = PyThreadState_New(mainInterpreterState); // free the lock PyEval_ReleaseLock();
Now that you have created a PyThreadState object, your C thread can begin to use the Python API to execute Python scripts. You must adhere to a few simple rules when executing Python code from a C thread. First, you must hold the global interpreter lock before doing anything that alters the state of the current thread state. Second, you must load your thread-specific PyThreadState object into the interpreter before executing any Python code. Once you have satisfied these constraints, you can execute arbitrary Python code by using functions such as PyEval_SimpleString. Remember to swap out your PyThreadState object and release the global interpreter lock when done. Note the symmetry of “lock, swap, execute, swap, unlock” in the code (ignore line wrapping):
// grab the global interpreter lock PyEval_AcquireLock(); // swap in my thread state PyThreadState_Swap(myThreadState); // execute some python code PyEval_SimpleString("import sys\n"); PyEval_SimpleString("sys.stdout.write('Hello from a C thread!\n')\n"); // clear the thread state PyThreadState_Swap(NULL); // release our hold on the global interpreter PyEval_ReleaseLock();
|Non-Linux FOSS: Install Windows? Yeah, Open Source Can Do That.||Nov 24, 2015|
|Cipher Security: How to harden TLS and SSH||Nov 23, 2015|
|Web Stores Held Hostage||Nov 19, 2015|
|diff -u: What's New in Kernel Development||Nov 17, 2015|
|Recipy for Science||Nov 16, 2015|
|Firefox's New Feature for Tighter Security||Nov 13, 2015|
- Non-Linux FOSS: Install Windows? Yeah, Open Source Can Do That.
- Cipher Security: How to harden TLS and SSH
- Simple Photo Editing, Linux Edition!
- Web Stores Held Hostage
- Firefox's New Feature for Tighter Security
- How Will the Big Data Craze Play Out?
- It's a Bird. It's Another Bird!
- diff -u: What's New in Kernel Development
- Libreboot on an x60, Part II: the Installation
- November 2015 Issue of Linux Journal: System Administration