Simple Access Berkeley DB Using STLdb4

The Berkeley DB library provides a solid implementation of both the B-Tree and Hash file structures. The implementation includes support for transactions, concurrent access of database files from multiple processes, and secondary indexing as well as logging and recovery.

In this article, I use the term database to refer to a B-Tree or Hash maintained by Berkeley DB. These databases allow rapid key to value look-ups.

The standard distribution of Berkeley DB comes with both a C and C++ API. Unfortunately, the standard Berkeley DB C++ API is a very thin wrapper neglecting modern C++ designs, such as smart pointers, standard C++ I/O streams, iterators, default arguments, operator overloading and so on. As a concrete example of the lack of reference counted smart pointers, the Berkeley DB API for Db::get(), shown in Listing 1, includes two Dbt pointers and the ownership of the memory for these is not immediately obvious.

#include <db_cxx.h>
int Db::get(DbTxn *txnid, Dbt *key, Dbt *data,
            u_int32_t flags);

The STLdb4 Project was created to make using the Berkeley DB from C++ easier. The STLdb4 API aims to make simple database interaction trivial while still keeping more advanced usage simple. A Berkeley DB object behaves similarly to an STL collection allowing look-ups and the setting of elements using an overloaded array operator. A full example program is shown in Listing 2. After execution, the file named with argv[1] will contain a Berkeley DB B-Tree file containing the foo-bar data pair.

The main class is the Database and the reference counted smart pointer for this class is called fh_database. This trend is used throughout STLdb4 where the smart pointer for Foo is called fh_foo. Databases can be opened either as in Listing 2 directly in the constructor or using the empty constructor and the open() or create() methods later. The main difference between open and create is that create requires a database type (for example B-Tree or Hash) and will create a new database at the given path if none exists already.

In the example in Listing 2, I don't have to close the database explicitly, because the smart pointer to the Database object will handle that for me.

#include <iostream>
#include <STLdb4/stldb4.hh>

using namespace STLdb4;
using namespace std;

int main( int argc, char** argv )
{
 fh_database db = new Database(DB_BTREE, argv[1]);
 db["foo"] = "bar";
 cerr << "foo is set to:" << db["foo"] << endl;
 return 0;
}

Standard STL collection methods, such as empty(), size(), insert(), erase(), count(), begin(), end(), find(), upper_bound() and lower_bound(), all exist in the Database class. There are also partial versions of the latter three methods. The partial versions allow the looking up of entries with part of a key in B-Tree files. A bidirectional iterator object is returned by many of the above methods.

When storing large values in the database, using the standard I/O streams can be more efficient than using the get() method or overloaded array operator. This is because the standard I/O streams use partial read and write operations on the underlying Berkeley DB file. A standard I/O stream is obtained using the getIStream() and getIOStream() methods of the Database class.

The example in Listing 3 shows the standard C++ I/O stream interface for STLdb4. The housekeeping of performing partial I/O to the Berkeley DB file is handled by STLdb4. Accessing large chunks of data through this API maintains a low memory consumption. The API shows one of the used getIOStream() calls as having a ferris_ios first parameter. As the libferrisstreams library that STLdb4 uses offers generic I/O stream support, the ferris_ios is a backward-compatible extension of the std::ios bitfield. The extension allows specifying such things as memory mapped backing and sequential stream access to be nominated for use where supported. The output from running this example is shown in Listing 4.