Asynchronous Database Access with Qt 4.x

How to code around the default synchronous database access in Qt 4.

Here, the MainWin and the QueryThread objects communicate directly with one another via signals and slots in the usual way. The trick here is that the QueryThread object, behind the scenes, utilizes a Worker object to perform all the work. Why is this extra level of indirection necessary? Because we want to dispatch the work to an object that is associated with a completely separate thread of execution. Notice above that QueryThread is instantiated and start()ed within the main thread of execution; therefore, QueryThread will “belong” to the application's main thread. Connecting signals from the application's widgets to its slots will be via “direct” connections—they will be invoked immediately by Qt, in the usual way, and thus be blocking, synchronous function calls. Instead, we are interested in “pushing” the signals into slots that are running in a separate thread of execution entirely, and this is where the internal Worker class comes into play (Listing 5).

Utilizing this Worker class internally, QueryThread can dispatch SQL queries properly to a separate thread by employing the convenient inter-thread signal/slot queued connections provided by Qt. QueryThread encapsulates the idea of a thread, by being derived from QThread and being able to start() a new thread of execution, and it also encapsulates the idea of a worker (by exposing convenient methods that perform database work, which are internally dispatched to a separate thread of execution). So, instead of tangling with all the necessary events and event handlers to accomplish the same task, the Qt metaobject system rigs up all the necessary code for you and exposes the connect() function to trigger it all. Signals can be emitted at any time and will be dispatched to the destination object correctly, within that object's context.

It is important to note that, above, the execute() method of QueryThread is intended to be invoked synchronously by the main application. QueryThread presents this method not only as a convenience; indeed, it also encapsulates and hides all the queued connection details from the user of the QueryThread class. When execute() is invoked, QueryThread simply emits it as a signal to the worker. Because the worker is an object “living” in a separate thread, Qt will “queue” the connection and pass it through the event loop so that it arrives in the proper execution context of the worker—essential so the worker can utilize its database connection according to the rules that are laid out in the beginning of this article. Finally, any signals coming from the worker are “forwarded” back out through the QueryThread interface—another convenience for the users of QueryThread, which also serves to hide all the details of the Worker class from those who really don't need to know about it in the first place.

If the sizes of your queries are gigantic, or potentially can be gigantic, you should consider a different strategy for dealing with the result sets. For example, writing the results out to disk before passing them back to the UI thread will reduce the memory load of your application, at the cost of some runtime speed. However, if you are already dealing with massive queries, the speed hit likely will be minor in comparison with the overall execution time of the query. Because all database queries execute in a completely separate thread from the UI, users will not perceive any significant lag. Store the intermediate results in a text file, or for maximum flexibility, in a local SQLite database.

One element of this puzzle still needs to be solved. The queued connection mechanism relies on the thread's event loop to invoke a slot within that thread's context properly, as discussed previously. However, in order to marshal the data necessary to dispatch the event to another thread of execution, the Qt object system must be made aware of any custom data types used in the signal/slot declaration. Failure to register custom data types will cause the queued connection to fail, because in order to dispatch the event, a copy of each of the signal's parameters must be made. Fortunately, it is a simple matter to “register” additional types, as long as those types have public constructors, copy constructors and destructors (Listing 6).

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

copying overhead?

Anonymous's picture

First off, great article Dave!

I have one remark though, and it concerns QByteArray QMetaObject::normalizedSignature ( const char * method ) and it's explanation:

“Qt uses normalized signatures to decide whether two given signals and slots are compatible. Normalization reduces whitespace to a minimum, moves 'const' to the front where appropriate, removes 'const' from value types and replaces const references with values.”

As far as I understand it, the signal will copy the whole result on each emit. Now, thats not a good idea for very large queries

So I wondered whether it would be better to create the result on the heap and to pass it's pointer with the signal? Would that introduce new complications?

Kind regards,
Davor

Thank you.

Andy Fillebrown's picture

Thanks for the article. I was looking into calling slots in a thread-safe manner and suspected correctly that the way to do it is via the QThread event loop. I did not realize, though, that it is possible to abstract the slots to a worker thread the way you described in the article. Knowing this now has saved me a lot of time.

Thanks again,
~ andy.f

Great article!

Antonio Menezes's picture

Thank you for your article. You have answered this question in Qt, about how to get asynchronous databases connections and operations.

Regards,
Antonio.

What about QT's MVC framework?

Peter's picture

Great article! I was very glad to find something that goes beyond the the basic Trolltech documentation.

Do you have any comments or advice regarding asynchronous DB access that takes advantage of QT's SQL model/view framework?

I have found that QTableView and QSqlQueryModel are wonderfully easy to work with except in the case when the GUI is blocked by while waiting for more complex queries to return their results.

I have played around with creating and executing a QSqlQuery inside a worker thread and then using a signal to pass the QSqlQuery object to the GUI thread when the worker thread finishes. However, this causes unpredictable crashes that I think are related to breaking the rule that connections must only be used in the threads that created them. (My QSqlQueryModel lives in the GUI thread, but the QSqlQuery that provides the model with data was created with a database connection that lives in the worker thread)

After reading your article I suspect that I should instead create a new model (subclass of QAbstractTableModel) that uses a QList for its data instead of a QSqlQuery.

Thanks for sharing any experience or comments!

Correct URL for sample application

dberton's picture

The correct URL for the sample application is ftp.linuxjournal.com/pub/lj/listings/issue158/9602.tgz

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState