Embedding Python in Your C Programs

Now, let's say that instead of having our expression calculator execute a list of expressions, you'd rather have it load a function f() from the Python file and execute it a variable number of times to calculate an aggregate total, based on a number provided on the command line. You could execute the function simply by running PyRun_SimpleString("f()"), but that's really not very efficient, as it requires the interpreter to parse and evaluate the string every time it's called. It would be much better if we could reference the function directly to call it.

If you recall, Python stores all globally defined functions in the global dictionary. Therefore, if you can get a reference to the global dictionary, you can extract a reference to any of the defined functions. Fortunately, the Python API provides functions for doing just that. You can see it in use by taking a look at Listing 5.

#include <python2.3/Python.h>

void process_expression(int num, char* func_name)
{
    FILE*        exp_file;
    PyObject*    main_module, * global_dict, * expression;

    // Initialize a global variable for
    // display of expression results
    PyRun_SimpleString("x = 0");

    // Open and execute the Python file
    exp_file = fopen(exp, "r");
    PyRun_SimpleFile(exp_file, exp);

    // Get a reference to the main module
    // and global dictionary
    main_module = PyImport_AddModule("__main__");
    global_dict = PyModule_GetDict(main_module);

    // Extract a reference to the function "func_name"
    // from the global dictionary
    expression =
        PyDict_GetItemString(global_dict, func_name);

    while(num--) {
        // Make a call to the function referenced
        // by "expression"
        PyObject_CallObject(expression, NULL);
    }
    PyRun_SimpleString("print x");
}

To obtain the function reference, the program first gets a reference to the main module by “importing” it using the PyImport_AddModule("__main__") function. Once it has this reference to the main module, the program uses the PyModule_GetDict() function to extract its dictionary. From there, it's simply a matter of calling PyDict_GetItemString(global_dict, "f") to extract the function from the dictionary.

Now that the program has a reference to the function, it can call it using the PyObject_CallObect() function. As you can see, this takes a pointer to the function object to call. Because the function itself already exists in the Python environment, it is already compiled. That means when you perform the call, there is no parsing and little or no compilation overhead, which means the function can be executed quite quickly.

At this point, I'm sure you're starting to think, “Gee whiz, this is great but it would be a whole lot better if I could actually pass some data to these functions I'm calling.” Well, you need wonder no longer. As it turns out, you can do exactly that. One way is through the use of that mysterious NULL value that you saw being passed to PyObject_CallObject in Listing 5. I'll talk about how that works in a bit, but first there is a much easier way to call functions with arguments that are in the form of C/C++ data types, PyObject_CallFunction(). Instead of requiring you to perform C-to-Python conversions, this handy function takes a format string and a variable number of arguments, much like the printf() family of functions.

Looking back at our calculator program, let's say you want to evaluate an expression over a range of noncontiguous values. If the expression to evaluate is defined in a function provided by the loaded Python file, you can get a reference as normal and then iterate over the range. For each value, simply call PyObject_CallFunction(expression, "i", num). The “i” string tells Python that you will be passing an integer as the only argument. If the function you were calling took two integers and a string instead, you could make the function call as PyObject_CallFunction(expression, "iis", num1, num2, string). If the function has a return value, it will be passed to you in the return value of PyObject_CallFunction(), as a PyObject pointer.

That's the easiest way to pass arguments to a Python function, but it's not actually the most flexible. Think about it for a second. What happens if you are dynamically choosing the function to call? The odds are that you're going to want the flexibility to call a variety of functions that accept different numbers and types of arguments. However, with PyObject_CallFunction(), you have to choose the number and type of the arguments at compile time, which hardly fits with the spirit of flexibility inherent in embedding a scripting language.

The solution is to use PyObject_CallObject() instead. This function allows you to pass a single tuple of Python objects instead of the variable-length list of native C data items. The downside here is that you will need to convert native C values to Python objects first, but what you lose in execution speed is made up for in flexibility. Of course, before you can pass values to your function as a Python tuple, you'll need to know how to create the tuple, which brings me to the next section.