Block Device Drivers: Interrupts

Last month, we gave an introduction to block device drivers. This month, we look at some tricks that are useful when writing block device drivers, starting with the most basic “trick” of using hardware interrupts where available and describing some neat infrastructure that block device drivers can take advantage of by adding five lines of code and one function.
Interrupts

I am going to take the foo driver I started developing last month, and add interrupt service to it. It is hard to write good, detailed code for a hypothetical and vaguely defined device, so (as usual) if you want to understand better after reading this, take a look at some real devices. I suggest the hd and floppy devices; start from the do_hd_request() and do_fd_request() routines and follow the logic through.

static void do_foo_request(void) {
  if (foo_busy)
    /* another request is being processed;
       this one will automatically follow */
    return;
    foo_busy = 1;
    foo_initialize_io();
}
static void foo_initialize_io(void) {
  if (CURRENT->cmd == READ) {
    SET_INTR(foo_read_intr);
  } else {
    SET_INTR(foo_write_intr);
  }
    /* send hardware command to start io
       based on request; just a request to
       read if read and preparing data for
       entire write; write takes more code */
}
static void foo_read_intr(void) {
  int error=0;
  CLEAR_INTR;
    /* read data from device and put in
       CURRENT->buffer; set error=1 if error
       This is actually most of the function... */
    /* successful if no error */
    end_request(error?0:1);
    if (!CURRENT)
      /* allow new requests to be processed */
      foo_busy = 0;
    /* INIT_REQUEST will return if no requests */
    INIT_REQUEST;
    /* Now prepare to do I/O on next request */
    foo_initialize_io();
}
static void foo_write_intr(void) {
        int error=0;
  CLEAR_INTR;
  /* data has been written. error=1 if error */
  /* successful if no error */
  end_request(error?0:1);
  if (!CURRENT)
    /* allow new requests to be processed */ foo_busy = 0;
/* INIT_REQUEST will return if no requests */
  INIT_REQUEST;
  /* Now prepare to do I/O on next request */
  foo_initialize_io();
}

In blk.h, we need to add a few lines to the FOO_MAJOR section:

#elif (MAJOR_NR == FOO_MAJOR)
#define DEVICE_NAME "foobar"
#define DEVICE_REQUEST do_foo_request
#define DEVICE_INTR do_foo
#define DEVICE_NR(device) (MINOR(device) > 6)
#define DEVICE_ON(device)
#define DEVICE_OFF(device)
#endif

Note that many functions are missing from this; this is the important part to understanding interrupt-driven device drivers; the “heart”, if you will. Also note that, obviously, I haven't tried to compile or run this hypothetical driver. I may have made some mistakes—you are bound to make mistakes of your own while writing your driver, and finding bugs in this skeleton will be good practice for finding bugs in your own driver, if you are so inclined. I do suggest that when you write your own driver, you start with code from some other working driver rather than starting from this skeleton code.

Structure

An explanation of some of the new ideas here is in order. The first new idea is (obviously, I hope) the use of interrupt routines to do part of servicing the hardware and walking down the request list. I used separate routines for reading and writing; this isn't fundamentally necessary, but it does generally help allow cleaner code and smaller, easier-to-read interrupt service routines. Most (all?) of the interrupt-driven device drivers of any kind in the real kernel use separate routines for reading and writing.

We also have a separate routine to do most of the I/O setup instead of doing it in the request() procedure. This is so that the interrupt routines can call the separate routings to set up the next request, if necessary, upon completion of a request. Again, this is a design feature that makes most real-world drivers smaller and easier to write and debug.

Context

It must be noted that any routine that is called from an interrupt is different than all the other routines I have described so far. Routines called from an interrupt do not execute in the context of any calling user-level program and cannot write to user-level memory. They can only write to kernel memory. If they absolutely need to allocate memory, they must do so with the GFP_ATOMIC priority. In general, it is best for them to write into buffers allocated from user-process-context routines with priority GFP_KERNEL before the interrupt routines are set up. They also can wake up processes sleeping on an event, as end_request() does, but they cannot sleep themselves.

Macros

The header file blk.h provides some nice macros which are used here. I won't document them all (most are documented in The Linux Kernel Hackers' Guide, the KHG), but I will mention the ones I use, which are used to manage interrupts.

Instead of setting up interrupts manually, it is easier and better to use the SET_INTR() macro. (If you want to know how to set them up manually, read the definitions of SET_INTR in blk.h.) Easier because you just do SET_INTR(interrupt_handling_function), and better because if you set up automatic timeouts (which we will cover later), SET_INTR() automatically sets them up.

Then, when the interrupt has been serviced, the interrupt service routine (foo_read_intr() or foo_write_intr() above) clears the interrupt, so that spurious interrupts don't get delivered to a procedure that thinks that it is supposed to read or write to the current request. It is possible—it only takes a little more work—to provide an interrupt routing to handle spurious interrupts. If you are interested, read the hd driver.

______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState