Dynamic Class Loading for C++ on Linux

A technique for developers that will provide them with much flexibility in design.

Linux has much to offer as a development platform: a robust operating environment with tested tools. Linux also boasts implementations of just about every programming language available. I think it is safe to say, however, that among compiled languages, C is the language of choice for most Linux developers. Consequently, other languages such as C++ seem to be somewhat neglected in most discussions of Linux development.

The dynamic class loading technique provides developers with a great deal of flexibility in their designs. Dynamic class loading is a means of providing extensibility without sacrificing robustness.

In this article, we will design a simple application that defines a single class, a shape class we wish to use in a drawing package. As we shall see, dynamic class loading allows us to provide a smooth extension path through which users of the application can add new types of shapes without needing to modify the original application code.

Polymorphism

The basic idea behind dynamic class loading is the concept of polymorphism. Anyone familiar with C++ should be familiar with this concept, so I will discuss it here only briefly. In short, polymorphism is the ability of an object belonging to a derived class to act as an object belonging to the base class. This is the familiar “is a” relationship of OOP (object-oriented programming) parlance. For example, in the following code snippet, circle is a class derived from the base class shape (see Listing 1), so the object my_circle can act as a shape object, invoking the shape member function draw.

Listing 1

class shape { public:
   void draw();
};
class circle : public shape { };
int main(int argc, char **argv){
   circle my_circle;
   my_circle.draw();
}

While this has all the usual advantages, e.g., code reuse, the real power of polymorphism comes into play when draw is declared to be virtual or pure virtual, as follows:

class shape{ public:
   virtual void draw()=0;
};
class circle : public shape { public:
   void draw();
}
Here, circle has declared its own draw function, which can define behavior appropriate for a circle. Similarly, we could define other classes derived from shape, which provide their own versions of draw. Now, because all the classes implement the shape interface, we can create collections of objects that can provide different behavior invoked in a consistent manner (calling the draw member function). An example of this is shown here.
shape *shape_list[3];   // the array that will
                            // pointer to our shape objects
shape[0] = new circle;  // three types of shapes
shape[1] = new square;  // we have defined
shape[2] = new triangle;
for(int i = 0; i < 3; i++){
   shape_list[i].draw();
}
When we invoke the draw function for each object on the list, we do not need to know anything about each object; C++ handles the details of invoking the correct version of draw. This is a very powerful technique, allowing us to provide extensibility in our designs. Now we can add new classes derived from shape to provide whatever behavior we desire. The key here is that we have separated the interface (the prototype for shape) from the implementation.

While this technique is very powerful, it does suffer from a drawback in that we must recompile (or at least relink) our code when we add new derived classes. It would be more convenient if we could simply load in new classes at runtime. Then, anyone using our code libraries could provide new shape classes (with new draw functions) without even needing our original source code. The good news, and the subject of this article, is that we can.

dlopen and Dynamic Class Loading

While C++ has no direct mechanism under Linux for loading in classes at runtime, there is a direct mechanism for loading C libraries at runtime: the dl functions dlopen, dlsym, dlerror and dlclose. These functions provide access to the dynamic linker ld. A complete description of these functions is provided in the appropriate man page, so they are presented here only briefly.

The prototypes for the functions are as follows:

void *dlopen(const char
void *dlsym(void *handle, char *symbol);
const char *dlerror();
int dlclose(void *handle);

The dlopen function opens the file given in filename so that the symbols in the file can be accessed via the dlsym function. flag can take one of two values: RTLD_LAZY or RTLD_NOW. If flag is set to RTLD_LAZY, dlopen returns without attempting to resolve any symbols. If flag is set to RTLD_NOW, dlopen attempts to resolve any undefined symbols in the file. Failure to resolve a symbol causes the call to fail, returning a NULL. dlerror can be used to provide an error message explaining the failure. The dlsym function is used to obtain a pointer to the functions (or other symbols) provided by the library. handle is the pointer to the thing being referenced, and symbol is the actual string name of the item referenced, as it is stored in the file.

Given that we can use these functions to access functions in a C library, how do we use them to access classes in a C++ library? There are several problems to overcome. One is that we must be able to locate the symbols we need in the library. This is trickier than it might seem because of the difference between the way symbols are stored in C and C++ files. Secondly, how can we create objects belonging to the classes we load? Finally, how can we access those objects in a useful manner? I will answer these three questions in reverse.

Since we do not have the prototypes for the classes we load dynamically, how can we access them in our code? The answer to this lies in the description of polymorphism in the preceding section. We access the functionality of the new classes through the common interface provided by their base class. Following the examples above, any new shape classes would provide a draw function that would allow an object of that class to render itself.

Fine; we can use pointers to the base class to access objects from the derived classes. How do we create the objects in the first place? We don't know anything about the classes that might be loaded, other than the fact that they conform to the shape interface. For instance, suppose we dynamically load a library that provides a class called hexapod. We can't write

shape *my_shape = new hexapod;

if we don't know the class name ahead of time.

The solution is that our main program doesn't create the objects, at least not directly. The same library that provides the class derived from shape must provide a way to create objects of the new class. This could be done using a factory class, as in the factory design pattern (see Resources) or more directly using a single function. To keep things simple, we will use a single function here. The prototype for this function is the same for all shape types:

shape *maker();

maker takes no arguments and returns a pointer to the constructed object. For our hexapod class, maker might look like this:

shape *maker(){
   return new hexapod;
}
It is perfectly legal for us to use new to create the object, since maker is defined in the same file as hexapod.

Now, when we use dlopen to load a library, we can use dlsym to obtain a pointer to the maker function for that class. We can then use this pointer to construct objects of the class. For example, suppose we want to dynamically link a library called libnewshapes.so which provides the hexapod class. We proceed as follows:

void *hndl = dlopen("libnewshapes.so", RTLD_NOW);
if(hndl == NULL){
   cerr << dlerror() << endl;
   exit(-1);
}
void *mkr = dlsym(hndl, "maker");

The pointer to maker must be of type void *, since that is the type returned by dlsym. Now we can create objects of the hexapod class by invoking mkr:

shape *my_shape = static_cast<shape *()>(mkr)();
We are required to cast mkr to a pointer to a function returning shape * when we invoke it.

Some readers may see a problem with the code as written thus far: the dlsym call is likely to fail because it cannot resolve "maker". The problem is that C++ function names are mangled to support function overloading, so the maker function may have a different name in the library. We could figure out the mangling scheme and search for the mangled symbol instead, but fortunately there is a much simpler solution. We need only tell the compiler to use C-style linkage using the extern "C" qualifier, as shown in Listing 2.

Listing 2

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

dependencies

Anonymous's picture

And what about depencies of dynamically loaded libs ?

I've got this pb :
I load dynamically some personal lib as you showed. But some objects of this lib also load some other lib dynamically. Those libs have direct dependencies with the root one , and when I dlopen them, I've got an error telling that dlopen tries to load the root lib (which is not in classical dirs, I loaded it with absolute path) and can't find it.

please if someone can help.

hi... i need to create the

Anonymous's picture

hi... i need to create the member variables in a class at runtime... also these variables should be defined by a config file... is it possible .. can anyone help ...

You mean shape *my_circle

Alejo's picture

You mean
shape *my_circle = factory["circle"]();
rather than
shape *my_circle = factory["circle"];

Nice article. :-)

Cpp

L S Srinivas Kothuri's picture

A very good finding for me. I developed a good amount of code in C but i'm new to cpp. I feel familiar with the dynamic libraries in cpp now. ThanQ.

Very useful

Greg Kogut's picture

This article was perfect, for not just learning basic dynamic module loading, but also the clever, clean technique for autoregistration. Everything worked beautifully for my project right away

Thank you for the article

John Gruenenfelder's picture

Great article and easy to follow. This is exactly the sort of information I was looking for. I've previously written a large application in C which used dynamic modules, but I've been digging around to find the "proper" way to do the same thing in C++. This article cut right to the chase.

Nice Article

Anonymous's picture

Hi ! Just wanted to say thanks for the nice article. Just tried it with g++ 4.xx. Works nearly out of the box... If you have more than one "shape" that you want to add, I would add static to the definitions of the maker function and the proxy p. This saves you from renaming it in every file.

It's all so wrong

Anonymous's picture

Factories, functions to access libraries, classes with textual names at runtime
It's all so wrong
C++ ! !
YECCCH !
I feel soiled just reading about it

And I feel dumber for having

Anonymous's picture

And I feel dumber for having read your comment. :-(

very constructive comment!

Anonymous's picture

probably from a .net zealot!
I actually like this article if only for the learning experience.

usage with Automake

bensch's picture

I tried to include the above code into our project using automake.
Since i am building the libs in -> src/entities/Makefile.am all the .so files go directly to src/entities/.libs/*.so
when i make a 'make install' the libs go to /usr/local/lib/*.so (or somewhere else)
now if my program in src/program wants to open the libs, it does not know in what path to search (i have to add it by using: dlopen("src/entities/.libs/module.so", RT_NOW);)

Since i think that Automake should provide a facility for the 'searchpath', and i am absolutely unable to find it:
DOES ANYONE know ho to do this??
any help is appreciated

Anybody could provide a

Anonymous's picture

Anybody could provide a working Makefile?, I can not understand how such a nice article can have this Makefile...

Makefile: You have to type

Anonymous's picture

Makefile: You have to type for each '=>'. Save this file as Makefile and run with make.

CC = g++
LIBS = -ldl

.cc.o:
=>$(CC) -ggdb -c $<

default:
=>make main

OBJS = testdcl.o

main: main.o
=>$(CC) -rdynamic -o main main.o $(LIBS)

libcircle.so: circle.o
=>g++ -shared -Wl,-soname,libcircle.so -o libcircle.so circle.o

libsquare.so: square.o
=>g++ -shared -Wl,-soname,libsquare.so -o libsquare.so square.o

all: main libcircle.so libsquare.so

clean:
=>rm -f *.so *.o main

management of vtable

Chetan Risbud's picture

Hi,

I just want to know the how does this manages the vtable after laoding the class dynamically ?

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Well, it's work fine for me (Gentoo Linux). But now I've tried to write a programm for a Solaris based system ... and the linker gives me some errors. So there must be a way to get it running, cause the compiler knows the dlopen(), dlsym() methods ... but the linker gives me errors.
Perhaps anybody can give me some hints ?
(Compiler : g++ 3.3)

Cross-platform loading of libraries.

Anonymous's picture

Is there any library that I can use to dynamically load C-libraries on LINUX and WINDOWs both?
I tried Libtool's libltdl library, but it doesnt seem to work on WINDOWS (VC++).

Re: Cross-platform loading of libraries.

Anonymous's picture

Eh? Windows has LoadLibrary() and GetProcAddress() which are nearly identical to dlopen() and dlsym() respectively.

Just make simple proxy functions that will use whichever method is appropriate for the proform it was compiled on.

Pseudo code:

LoadDynamicLibrary(char *path)
{
#ifdef _WIN32
   LoadLibrary(path);
#else
   dlopen(path);
#endif
}

can you create an object (of

Anonymous's picture

can you create an object (of a class in the dll) using GetProcAddress? pls help...

Re: Cross-platform loading of libraries.

Anonymous's picture

If you're writing a Qt application, you have the QLibrary class that OS independent resolves symbols from .so's resp. .dll's.

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

FYI you cannot throw, with ELF/linux under gcc 3.2.3, within the autoregistration proxy class constructor. So your constructor, as in the above example, is implicitly:

class proxy {

public:

proxy() throw() {

factory["shape name"] = maker;

}

};

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

It's a good article!!

I can compile your codes wih gcc-2.96 (rh7.2), but no luck with gcc-3.2. I'm a newbir on C++. So, can you make a version that can be compiled with gcc-3.2.

Thx

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Oh yeah, almost forgot, you also want to make sure you don't forget to add the -ldl flag as a g++ command line argument. That tells the linker it should load the library you need to execute the dlxxxx series calls.

Cheers,

tundog

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

I had some trouble using gcc-3.2 as well. I couldn't get it to work with the static_cast syntax.

Instead of:

shape *my_shape = static_cast(mkr)();

Try using:

shape *my_shape = ((shape*(*)())(mkr))();

Cheers,

tundog

You can use reinterpret_cast

Anonymous's picture

You can use reinterpret_cast instead of static_cast. This works for me.

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

XPLC does all of this, faster, better, simpler, safer.

http://xplc.sourceforge.net/

xplc is 0.3.0 Alpha

Anonymous's picture

I presume that's why more stable is missing on your list of properties...

Re: xplc is 0.3.0 Alpha

Anonymous's picture

Well, this is just some dlopen() sample code in an article, it's not like it's exactly a solid framework either...

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

bull***** it does.. lol

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Is there any particular reason why deleting objects is omitted?

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Wow, this works great, but I did run into one problem. I have a shape class which declares a draw() method. I can dynamically load the circle and square classes (which inherit from shape) and get polymorphic behavior. However, this only works if shape defines draw() as an empty method as opposed to a pure virtual method. If I try to declare draw() as a pure method I get an error during dlopen() that says "undefined symbol: __pure_virtual".

Is there a way I can get around this?

Thanks

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

That factory is nice idea but there are some problems with it:

  • It can be slow.
  • It requires extern "C" functions _init() and _fini() in the library. And in dynamic loader there is a serious bug: if you use any of these functions then global static objects are not constructed (i.e. their constructor is not called) so they are in undefined state.

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

While most of what you say is applicable to my situation, there is one main assumption that you've used... The base class has to be fixed, and dynamically loaded class has no knowledge of the classes in which it has just been instantiated with.

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

#######################

Error while executing the above program of shape :::

In function `int main ()':

m.C:22: cannot dynamic_cast `mkr' (of type `void *') to type `class

shape *' (source is not a pointer to class)

Can any one help me ????

I found out what the problem

TeacherDon's picture

I found out what the problem is. POSIX requires dlsym to return an object pointer while C++ requres that object pointers cannot be cast into function pointers. (This is because some systems have different pointer sizes for objects and functions.) The following union hack should work.


template<class TYPE>
TYPE function_cast(void * symbol)
{
    assert(sizeof(void *) == sizeof(TYPE));
    union
    {
        void * symbol;
        TYPE function;
    } cast;
    cast.symbol = symbol;
    return cast.function;
}

Then you should be able to cast to any pointer-to-function type, for exampe:


shape * my_shape = function_cast<shape(*)()>(mkr)();

A minor correction:

Anonymous's picture

A minor correction:

shape * my_shape = function_cast(mkr)();

A correction of my

Anonymous's picture

A correction of my correction:
shape * my_shape = function_cast(mkr)();

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Hi,

You can try this way :

shape *(*mkr)()= (shape*(*)())dlsym(dlib, "maker");
shape *my_shape = (*mkr)();
my_shape->draw();

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

I too had alot of trouble getting this to compile. So I got the mini-dlopen example to work. Then, I came back to this one and it worked immediately after I re-downloaded it. So keep persevering!

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

Yes.

it sounds like your compiler is interpreting this line:

shape *my_shape = static_cast(mkr)();

as "cast 'mkr' to 'class shape *' " which is indeed wrong.

you want to cast 'mkr', which is of type pointer-to-function-returning-void*, to type pointer-to-function-returning-shape*, then call it.

Perhaps extracting the function pointer types out to typedefs would help you/the compiler clarify what is needed.

Or maybe just calling mkr(), then casting the returned pointer from void* to shape* would work equally well?

hth,

paulmg

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

#######################

Error while executing the above program of shape :::

In function `int main ()':

m.C:22: cannot dynamic_cast `mkr' (of type `void *') to type `class

shape *' (source is not a pointer to class)

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

I believe there is a missing

"pclose(dl);" statement.

No big deal.. but running this 'pluggin' framework,

in a loop causes a memory leak then.

Kind Regards,

Unk

--

http://www.triple-it.nl

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

I would also replace the command with

dlib = dlopen(name, RTLD_NOW);

by

dlib = dlopen(name, RTLD_NOW|RTLD_GLOBAL);

So symbols from one Shared Library Object file are avaible

to an other Shared Library Object file.

Unk

--

http://www.triple-it.nl/

Re: Dynamic Class Loading for C++ on Linux

Anonymous's picture

I forget to mention that you should rename all the

maker() and proxy-class names.. to prevent name-clashes. (Otherwise you instantiate wrong classes)

Kind Regards,

Unk

--

http://www.triple-it.nl/

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState