Embedding Python in Your C Programs

// Get a reference to the main module.
PyObject* main_module =
   PyImport_AddModule("__main__");

// Get the main module's dictionary
// and make a copy of it.
PyObject* main_dict =
   PyModule_GetDict(main_module);
PyObject* main_dict_copy =
   PyDict_Copy(main_dict);

// Execute two different files of
// Python code in separate environments
FILE* file_1 = fopen("file1.py", "r");
PyRun_File(file_1, "file1.py",
           Py_file_input,
           main_dict, main_dict);

FILE* file_2 = fopen("file2.py", "r");
PyRun_File(file_2, "file2.py",
           Py_file_input,
           main_dict_copy, main_dict_copy);

I'll get into the details of how PyRun_File() works in a little bit, but if you look carefully at Listing 3, you should notice something interesting. When I call PyRun_File() to execute the files, the dictionary gets passed in twice. The reason for this is that Python code actually has two environmental contexts when it is executed. The first is the global context, which I've already talked about. The second context is the local context, which contains any locally defined variables or functions. In this case, those are the same, because the code being executed is top-level code. On the other hand, if you were to execute a function dynamically using multiple C-level calls, you might want to create a local context and use that instead of the global dictionary. For the most part though, it's generally safe to pass the global environment for both the global and local parameters.

At this point, I'm sure you've noticed the Py_DECREF() calls that popped up in the Listing 3 example. Those fun little guys are there for memory management purposes. Inside the interpreter, Python handles memory management automatically by keeping track of all references to memory transparent to the programmer. As soon as it determines that all references to a given chunk of memory have been released, it deallocates the no-longer needed chunk. This can be a problem when you start working on the C side though. Because C is not a memory-managed language, as soon as a Python data structure ends up referenced from C, all ability to track the references automatically is lost to Python. The C application can make as many copies of the reference that it wants, and hold on to it indefinitely without Python knowing anything about it.

The solution is to have C code that gets a reference to a Python object handle all of the reference counting manually. Generally, when a Python call hands an object out to a C program, it increments the reference count by one. The C code can then do what it likes with the object without worrying that it will be deleted out from under it. Then when the C program is done with the object, it is responsible for releasing its reference by making a call to Py_DECREF().

It's important, though, to remember when you copy a pointer within your C program that may outlast the pointer from which you're copying, you need to increment the reference count manually, by calling Py_INCREF(). For example, if you make a copy of a PyObject pointer to store inside an array, you'll probably want to call Py_INCREF() to ensure that the pointed-to object won't get garbage-collected after the original PyObject reference is decremented.

Now let's take a look at a slightly more useful example to see how Python can be embedded into a real program. If you take a look at Listing 4, you'll see a small program that allows the user to specify short expressions on the command line. The program then calculates the results of those expressions and displays them in the output. To add a little spice to the mix, the program also lets users specify a file of Python code that will be loaded before the expressions are executed. This way, the user can define functions that will be available to the command-line expressions.

#include <python2.3/Python.h>

void process_expression(char* filename,
                        int num,
                        char** exp)
{
    FILE*       exp_file;

    // Initialize a global variable for
    // display of expression results
    PyRun_SimpleString("x = 0");

    // Open and execute the file of
    // functions to be made available
    // to user expressions
    exp_file = fopen(filename, "r");
    PyRun_SimpleFile(exp_file, exp);

    // Iterate through the expressions
    // and execute them
    while(num--) {
        PyRun_SimpleString(*exp++);
        PyRun_SimpleString("print x");
    }
}

int main(int argc, char** argv)
{
    Py_Initialize();

    if(argc != 3) {
        printf("Usage: %s FILENAME EXPRESSION+\n");
        return 1;
    }
    process_expression(argv[1], argc - 1, argv + 2);
    return 0;
}

Two basic Python API functions are used in this program, PyRun_SimpleString() and PyRun_AnyFile(). You've seen PyRun_SimpleString() before. All it does is execute the given Python expression in the global environment. PyRun_SimpleFile() is similar to the PyRun_File() function that I discussed earlier, but it runs things in the global environment by default. Because everything is run in the global environment, the results of each executed expression or group of expressions will be available to those that are executed later.