Asynchronous Database Access with Qt 4.x

How to code around the default synchronous database access in Qt 4.

The pseudo-code in Listing 2 creates and starts two worker threads. Each thread establishes a named connection to the database and waits for work to do. Each thread always will deal only with its own database connection, which is never shared or visible outside the worker thread object.

Setting up thread-specific connections and running queries within that thread is only the first part of the problem. A decision needs to be made regarding how data is shuffled back and forth between the worker threads and the main application UI thread. Additional methods will give the thread object some database work to perform, and those methods will themselves need to be thread-safe.

Moving Data between Threads

You should observe all of the usual caveats surrounding the sharing of data between different threads of execution, including the proper uses of mutexes and wait conditions. In addition, there is an added complication regarding the size of the data, which potentially can be extremely large in the case of result sets returned from the database.

Qt provides several specialized mechanisms for sending data between threads. You can post events manually to any object in any thread using the thread-safe function QCoreApplication::postEvent(). The events will be dispatched automatically by the event loop of the thread where the destination object was created. To create an event loop within each thread, use QThread::exec(). Using this method, threads are given “work” to do in the form of events. QCoreApplication passes these events in a thread-safe manner from the application thread to the worker thread, and they will be handled by the worker thread within its own execution context. This is critical, because the worker thread will be utilizing its database connection only from within its own context (Listing 3).

This method works in the other direction as well. Events posted by individual worker threads will show up back in the main UI event loop for handling, within the context of the UI's thread of execution. These events can, for example, contain the result of a query or database update. This approach is convenient, as the worker threads can simply “post it and forget it”, and Qt will take care of the inter-thread communication, mutexing and memory management for you.

Constructing all the necessary events and event handlers for this type of system has advantages—most notably compile-time type checking. However, getting all the events designed and dealt with properly can be an onerous task, especially if your application has multiple types of database queries to perform, with multiple return types, each of which needing an associated event and event handler and so forth.

Fortunately, Qt permits signals and slots to be connected across threads—as long as the threads are running their own event loops. This is a much cleaner method of communication compared to sending and receiving events, because it avoids all the bookkeeping and intermediate QEvent-derived classes that become necessary in any nontrivial application. Communicating between threads now becomes a matter of connecting signals from one thread to the slots in another, and the mutexing and thread-safety issues of exchanging data between threads are handled by Qt.

Why is it necessary to run an event loop within each thread to which you want to connect signals? The reason has to do with the inter-thread communication mechanism used by Qt when connecting signals from one thread to the slot of another thread. When such a connection is made, it is referred to as a queued connection. When signals are emitted through a queued connection, the slot is invoked the next time the destination object's event loop is executed. If the slot had instead been invoked directly by a signal from another thread, that slot would execute in the same context as the calling thread. Normally, this is not what you want (and especially not what you want if you are using a database connection, as the database connection can be used only by the thread that created it). The queued connection properly dispatches the signal to the thread object and invokes its slot in its own context by piggy-backing on the event system. This is precisely what we want for inter-thread communication in which some of the threads are handling database connections. The Qt signal/slot mechanism is at root an implementation of the inter-thread event-passing scheme outlined above, but with a much cleaner and easier-to-use interface.

For example, you can create two simple connections between the main UI object and a worker thread object; one to add a query to the worker thread and another to report back the results. This simple setup, only a few lines of code, establishes the main communication mechanism for an asynchronous database application (Listing 4).

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

copying overhead?

Anonymous's picture

First off, great article Dave!

I have one remark though, and it concerns QByteArray QMetaObject::normalizedSignature ( const char * method ) and it's explanation:

“Qt uses normalized signatures to decide whether two given signals and slots are compatible. Normalization reduces whitespace to a minimum, moves 'const' to the front where appropriate, removes 'const' from value types and replaces const references with values.”

As far as I understand it, the signal will copy the whole result on each emit. Now, thats not a good idea for very large queries

So I wondered whether it would be better to create the result on the heap and to pass it's pointer with the signal? Would that introduce new complications?

Kind regards,
Davor

Thank you.

Andy Fillebrown's picture

Thanks for the article. I was looking into calling slots in a thread-safe manner and suspected correctly that the way to do it is via the QThread event loop. I did not realize, though, that it is possible to abstract the slots to a worker thread the way you described in the article. Knowing this now has saved me a lot of time.

Thanks again,
~ andy.f

Great article!

Antonio Menezes's picture

Thank you for your article. You have answered this question in Qt, about how to get asynchronous databases connections and operations.

Regards,
Antonio.

What about QT's MVC framework?

Peter's picture

Great article! I was very glad to find something that goes beyond the the basic Trolltech documentation.

Do you have any comments or advice regarding asynchronous DB access that takes advantage of QT's SQL model/view framework?

I have found that QTableView and QSqlQueryModel are wonderfully easy to work with except in the case when the GUI is blocked by while waiting for more complex queries to return their results.

I have played around with creating and executing a QSqlQuery inside a worker thread and then using a signal to pass the QSqlQuery object to the GUI thread when the worker thread finishes. However, this causes unpredictable crashes that I think are related to breaking the rule that connections must only be used in the threads that created them. (My QSqlQueryModel lives in the GUI thread, but the QSqlQuery that provides the model with data was created with a database connection that lives in the worker thread)

After reading your article I suspect that I should instead create a new model (subclass of QAbstractTableModel) that uses a QList for its data instead of a QSqlQuery.

Thanks for sharing any experience or comments!

Correct URL for sample application

dberton's picture

The correct URL for the sample application is ftp.linuxjournal.com/pub/lj/listings/issue158/9602.tgz

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix