Asynchronous Database Access with Qt 4.x

How to code around the default synchronous database access in Qt 4.

Here, the MainWin and the QueryThread objects communicate directly with one another via signals and slots in the usual way. The trick here is that the QueryThread object, behind the scenes, utilizes a Worker object to perform all the work. Why is this extra level of indirection necessary? Because we want to dispatch the work to an object that is associated with a completely separate thread of execution. Notice above that QueryThread is instantiated and start()ed within the main thread of execution; therefore, QueryThread will “belong” to the application's main thread. Connecting signals from the application's widgets to its slots will be via “direct” connections—they will be invoked immediately by Qt, in the usual way, and thus be blocking, synchronous function calls. Instead, we are interested in “pushing” the signals into slots that are running in a separate thread of execution entirely, and this is where the internal Worker class comes into play (Listing 5).

Utilizing this Worker class internally, QueryThread can dispatch SQL queries properly to a separate thread by employing the convenient inter-thread signal/slot queued connections provided by Qt. QueryThread encapsulates the idea of a thread, by being derived from QThread and being able to start() a new thread of execution, and it also encapsulates the idea of a worker (by exposing convenient methods that perform database work, which are internally dispatched to a separate thread of execution). So, instead of tangling with all the necessary events and event handlers to accomplish the same task, the Qt metaobject system rigs up all the necessary code for you and exposes the connect() function to trigger it all. Signals can be emitted at any time and will be dispatched to the destination object correctly, within that object's context.

It is important to note that, above, the execute() method of QueryThread is intended to be invoked synchronously by the main application. QueryThread presents this method not only as a convenience; indeed, it also encapsulates and hides all the queued connection details from the user of the QueryThread class. When execute() is invoked, QueryThread simply emits it as a signal to the worker. Because the worker is an object “living” in a separate thread, Qt will “queue” the connection and pass it through the event loop so that it arrives in the proper execution context of the worker—essential so the worker can utilize its database connection according to the rules that are laid out in the beginning of this article. Finally, any signals coming from the worker are “forwarded” back out through the QueryThread interface—another convenience for the users of QueryThread, which also serves to hide all the details of the Worker class from those who really don't need to know about it in the first place.

If the sizes of your queries are gigantic, or potentially can be gigantic, you should consider a different strategy for dealing with the result sets. For example, writing the results out to disk before passing them back to the UI thread will reduce the memory load of your application, at the cost of some runtime speed. However, if you are already dealing with massive queries, the speed hit likely will be minor in comparison with the overall execution time of the query. Because all database queries execute in a completely separate thread from the UI, users will not perceive any significant lag. Store the intermediate results in a text file, or for maximum flexibility, in a local SQLite database.

One element of this puzzle still needs to be solved. The queued connection mechanism relies on the thread's event loop to invoke a slot within that thread's context properly, as discussed previously. However, in order to marshal the data necessary to dispatch the event to another thread of execution, the Qt object system must be made aware of any custom data types used in the signal/slot declaration. Failure to register custom data types will cause the queued connection to fail, because in order to dispatch the event, a copy of each of the signal's parameters must be made. Fortunately, it is a simple matter to “register” additional types, as long as those types have public constructors, copy constructors and destructors (Listing 6).

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

copying overhead?

Anonymous's picture

First off, great article Dave!

I have one remark though, and it concerns QByteArray QMetaObject::normalizedSignature ( const char * method ) and it's explanation:

“Qt uses normalized signatures to decide whether two given signals and slots are compatible. Normalization reduces whitespace to a minimum, moves 'const' to the front where appropriate, removes 'const' from value types and replaces const references with values.”

As far as I understand it, the signal will copy the whole result on each emit. Now, thats not a good idea for very large queries

So I wondered whether it would be better to create the result on the heap and to pass it's pointer with the signal? Would that introduce new complications?

Kind regards,
Davor

Thank you.

Andy Fillebrown's picture

Thanks for the article. I was looking into calling slots in a thread-safe manner and suspected correctly that the way to do it is via the QThread event loop. I did not realize, though, that it is possible to abstract the slots to a worker thread the way you described in the article. Knowing this now has saved me a lot of time.

Thanks again,
~ andy.f

Great article!

Antonio Menezes's picture

Thank you for your article. You have answered this question in Qt, about how to get asynchronous databases connections and operations.

Regards,
Antonio.

What about QT's MVC framework?

Peter's picture

Great article! I was very glad to find something that goes beyond the the basic Trolltech documentation.

Do you have any comments or advice regarding asynchronous DB access that takes advantage of QT's SQL model/view framework?

I have found that QTableView and QSqlQueryModel are wonderfully easy to work with except in the case when the GUI is blocked by while waiting for more complex queries to return their results.

I have played around with creating and executing a QSqlQuery inside a worker thread and then using a signal to pass the QSqlQuery object to the GUI thread when the worker thread finishes. However, this causes unpredictable crashes that I think are related to breaking the rule that connections must only be used in the threads that created them. (My QSqlQueryModel lives in the GUI thread, but the QSqlQuery that provides the model with data was created with a database connection that lives in the worker thread)

After reading your article I suspect that I should instead create a new model (subclass of QAbstractTableModel) that uses a QList for its data instead of a QSqlQuery.

Thanks for sharing any experience or comments!

Correct URL for sample application

dberton's picture

The correct URL for the sample application is ftp.linuxjournal.com/pub/lj/listings/issue158/9602.tgz

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix