Block Device Drivers: Interrupts

Block devices, which are usually intended to hold file-systems, may or may not be interrupt-driven. Interrupt-driven block device drivers have the potential to be faster and more efficient than non- interrupt-driven block device drivers.

Last month, I gave an example of a very simplistic block device driver that reads its request queue one item at a time, satisfying each request in turn, until the request queue is emptied, and then returning. Some block device drivers in the standard kernel are like this. The ramdisk driver is the obvious example; it does very little more than the simplistic block device driver I presented. Less obvious to the casual observer, few of the CD-ROM drivers (actually none of them, as I write this) are interrupt-driven. It is easy to determine which drivers are interrupt-driven by reading drivers/block/blk.h, searching for the string DEVICE_INTR, and noting which devices use it.

I'm tired of typing “block device driver”, and you are probably tired of reading it. For the rest of this article, I will use “driver” to mean “block device driver”, except where stated otherwise.

Interrupt-driven drivers have the potential to be more efficient than non-interrupt-driven ones because the drivers have to spend less time busy-waiting—sitting in a tight loop, waiting for the device to become ready or finish executing a command. They also have the potential to be faster, because it may be possible to arrange for multiple requests to be satisfied at once, or to take advantage of peculiarities of the hardware.

In particular, the SCSI disk driver tries to send the SCSI disk one command to read multiple sectors and satisfy each of the requests as the data for each block arrives from the disk. This is a big win considering the way the SCSI interface is designed; because initiating a SCSI transfer takes some complex negotiation, it takes a significant amount of time to negotiate a SCSI transfer, and when the SCSI driver can ask for multiple blocks at the same time, it only has to negotiate the transfer once, instead of once for each block.

This complex negotiation makes SCSI a robust bus that is useful for many things besides disk drives. It also makes it necessary to pay attention to timing when writing the driver, in order to take advantage of the possibilities without being extremely slow. Before certain optimizations were added to the generic, high-level SCSI driver in Linux, SCSI performance did not at all approach its theoretical peak. Those optimizations made for throughput 3 to 10 times greater on most devices.

As another example, the original floppy driver in Linux was very slow. Each time it wanted a block, it read it in from the media. The floppy hardware is very slow and has high latency (it rotates slowly and if you wanted to read the block that just started going past the head, you had to wait until the disk made a full revolution), which kept it very slow.

Around version .12, Lawrence Foard added a track buffer. Since it only takes approximately 30% to 50% more time to read an entire track off the floppy as it does to wait for the block you want to read to come around and be read (depending on the type of disk and the position of the disk at the start of the request), it makes sense, when reading a block, to read the entire track the block is in.

As soon as the requested block has been read into the track buffer, it is copied into the request buffer, the process waiting for it to be read can continue, and the rest of the track is read into a private buffer area belonging to the floppy driver. The next request for a block from that floppy is often for the very next block, and that block is now in the track buffer and ready immediately to be used to fulfill the request. This is true approximately 8 times out of 9 (assuming 9 blocks, or 18 sectors, per track). This single change turned the floppy driver from a very slow driver into a very fast driver.

So, you are convinced that interrupt-driven drivers have a lot more potential, and you want to know how to turn the non-interrupt-driven driver you wrote last month into an interrupt-driven one. I can't give you all the information you need in a single article, but I can get you started, and after reading the rest of this article, you will be better prepared to read the source code for real drivers, which is the best preparation for writing your own driver.

The basic control flow of a request for a block from a non-interrupt-driven driver usually runs something like this simplification alert:

user program calls read() read() (in the kernel) asks the buffer cache to get and fill in the block buffer cache notices that it doesn't have the data in the cache buffer cache asks driver to fill in a block with correct data driver satisfies request and returns buffer cache passes newly-filled-in block back to read() read() copies the data into the user program and returns user program continues An interrupt-driven driver runs more like this simplification alert: user program calls read() read() (in the kernel) asks the buffer cache to get and fill in the block buffer cache notices that it doesn't have the data in the cache buffer cache asks driver to fill in a block with correct data driver starts the process of satisfying the request and returns buffer cache waits for block to be read by sleeping on an event Some other processes run for a while, perhaps causing other I/O on the device. the physical device has the data available and interrupts the driver driver reads the data from the device and wakes up the buffer cache buffer cache passes the newly-filled-in block back to read(). read() copies the data into the user program and returns user program continues

Note that read() is not the only way to initiate I/O.

One thing to note about this is that just about anything can be done before waking up the process(es) waiting for the request to complete. In fact, other requests might be added to the queue. This seems, at first, like a troublesome complication, but really is one of the important things that makes it possible to do some worthwhile optimizations. This will become obvious as we start to optimize the driver. We will start, though, by taking our non-interrupt-driven driver and making it use interrupts.