Portable Database Management with /rdb

The /rdb database system from Revolutionary Software is a compact, low-cost relational database management system that has been available for Linux and other Unix variants for several years. It distinguishes itself from most (if not all) other Unix database systems by functioning as an extension of the Unix shell.

The typical Unix distribution contains a rich set of shell commands (often referred to as filters) such as awk, grep and sed that operate primarily on text data. In addition, pipes can be used to route the output from one command into the input of another command. These commands can either be standard commands like those mentioned above or be created by the user. This system of pipes and filters simplifies the development of complex applications by enabling the reuse of components that have already been tested and proven.

Most filters are designed to operate on ASCII text data, so it is natural to represent logical data records as ASCII text lines. Data elements within the records are separated by a field delimiter character (such as a colon or tab) specified by the application. Alphanumeric data elements such as names, addresses and descriptions can be as long or as short as needed. Numeric data elements can have unlimited magnitude and precision, and their representation is architecture independent. This scheme provides complete portability of data between platforms.

Approaches to application design inherited from other platforms typically use fixed-length fields with predetermined field lengths. These lengths result in wasted file space due to the unpredictability of an application's actual storage requirements. Numeric data is often stored in binary format which makes it highly platform specific and nonportable. Data elements such as “description” or “comment” fields can vary greatly in size, and the use of fixed-length fields forces the application designer to either allocate enough storage to cover all cases (often impossible) or to dedicate separate data files to their storage (often making application logic clumsy, complex and difficult to manage).

The result of these approaches is the allocation of disk space that is never used and application programs that are difficult to maintain. Although recent innovations such as SQL have simplified the problem of data access from applications, many programs are tightly bound to storage layouts determined at compile time. This binding of code to data forces recompilation and relinking of application code each time a change in record format is required. Worse yet, many applications use “filler fields” to avoid recompilation, wasting even more disk space and I/O bandwidth.

While a great many Unix systems (database systems included) use the approach outlined in the last section, /rdb augments the capabilities already provided by standard Unix shell commands. Many /rdb commands operate as front ends for standard Unix commands such as awk. This enables users already familiar with Unix to create /rdb applications without learning a new programming language.

The format of /rdb tables is extremely simple. They are ASCII text files in which a row is a line of data. The column elements within each row are delimited by a column separation character (a tab by default but any other character except the newline can be used). /rdb commands use a -f command line switch to specify a column separator in cases where it is not a tab. Note the similarity between this approach and typical Unix system configuration data files such as /etc/passwd. The only difference is that the first line of /rdb tables is a delimited set of field names and the second line is a line of dashes. These first two lines are known as a header.

Part of the beauty of /rdb tables is that, for some applications, they need not even be stored in disk files. Instead, they can be passed between commands using pipes. These commands can be /rdb commands or standard Unix shell commands. The /rdb commands are designed to output tables through stdout, and most accept input tables through stdin. Although some standard Unix commands will be “confused” by the header lines, /rdb provides commands for removing and reattaching these headers (headoff and headon, respectively).