The Devil's in the Details

Once upon a time, a beautiful princess was sent by a witch into a long, deep sleep, lasting for a hundred years. The world nearly forgot her and her castle, twined about by roses, until one day, a handsome prince came, kissed her, and awakened her --and all the other nice things happened that you hear about in fairy tales.

Our driver should do what the princess did while it is waiting for data: sleep, leaving the world spinning around. Linux provides a mechanism for that, called interruptible_sleep_on(). Every process reaching this call will fall asleep and contribute its time slices to the rest of the world. It will stay in this function till another process calls wake_up_interruptible(), and this “prince” usually takes the form of an interrupt handler that has successfully received or sent data, or Linux itself, if a time-out condition has occurred.

The previous article in this series showed a minimal interrupt handler, which was called skel_trial_fn(), but its workings were not explained. Here, we introduce a “complete” interrupt handler, which will handle both input to and output from the actual hardware device. Figure 1 shows a simple version of its concept: When the driver is waiting for the device to get ready (blocking), it goes to sleep by calling interruptible_sleep_on(). A valid interrupt ends this sleep, restarting skel_write().

Figure 1 does not include the double-nested loop structure we need when working with an internal output buffer. The reason is that if we can perform only writing within the skel_write() function there is no need for an internal output buffer. But our driver should catch data even while not in skel_read() and should write the data in the background even when not in skel_write(). Therefore, we will change the hardware writing in skel_write() to write to an output buffer and let the interrupt handler perform the real writing to the hardware. The interrupt and skel_write() will now be linked by the “Sleeping Beauty” mechanism and the output buffer.

The interrupt handler is installed and uninstalled during the open() and close() calls to the device, as suggested in the previous article. This task is handled by the following kernel calls:

#include <linux/sched.h>
int request_irq(unsigned int irq,
                void (*handler)
                     (int, struct pt_regs *),
                unsigned long flags,
                const char *device);
void free_irq(unsigned int irq);

The handler argument is the actual interrupt handler we wish to install. The role of the flags argument is to set a few features of the handler, the most important being its behaviour as a fast handler (SA_INTERRUPT is set in flags) or as a slow handler ( SA_INTERRUPT is not set). A fast handler is run with all interrupts disabled, while a slow one is executed with all interrupts except itself enabled.

Finally, the device argument is used to identify the handler when looking at /proc/interrupts.

The handler function installed by request_irq() is passed only the interrupt number and the (often useless) contents of the processor registers.

Therefore, we'll first determine which board the calling interrupt belongs to. If we can't find any boards, a situation called a spurious interrupt has occurred, and we should ignore it. Typically interrupts are used to tell whether the device is ready either for reading or writing, so we have to find out by one or more hardware tests what the device wants us to do.

Of course, we should leave our interrupt handler quickly. Strangely enough, printk() (and thus the PDEBUG line) is allowed even within fast interrupt handlers. This is a very useful feature of the linux implementation. If you look at kernel/printk.c you'll discover that its implementation is based on wait queues, as the actual delivery of messages to log files is handled by an external process (usually klogd).

As shown in figure 2, Linux can handle a timeout when in interruptible_sleep_on(). For example, if you have are using a device to which you send an answer, and it is expected to reply within a limited time, causing a time-out to signal an I/O error (-EIO) in the return value to the user process might be a good choice.

Certainly the user process could care for this, too, using the alarm mechanism. But it is definitely easier to handle this in the driver itself. The timeout criteria is specified by SKEL_TIMEOUT, which is counted in jiffies. Jiffies are the steady heartbeat of a Linux system, a steady timer incremented every few milliseconds. The frequency, or number of jiffies per second, is defined by HZ in <asm/param.h> (included in <linux/sched.h>) and varies on different architectures (100 Hz Intel, 1 kHz Alpha). You simply have to set

#define SKEL_TIMEOUT timeout_seconds * HZ
/* ... */
current->timeout = jiffies + SKEL_TIMEOUT

and if interruptible_sleep_on timed out, current->timeout will be cleared after return.

Be aware that interrupts might happen within skel_read() and skel_write(). Variables that might be changed within the interrupt should be declared as volatile. They also need to be protected to avoid race conditions. The classic code sequence to protect a critical region is the following:

unsigned long flags;
save_flags (flags);
cli ();
critical region
restore_flags (flags);

Finally, the code for the “complete” error handler:

#define SKEL_IBUFSIZ 512
#define SKEL_OBUFSIZ 512
/* for 5 seconds timeout */
#define SKEL_TIMEOUT (5*HZ)
/* This should be inserted in the Skel_Hw-structure */
typedef struct Skel_Hw {
    /* write position in input-buffer */
    volatile int ibuf_wpos;
    /* read position in input-buffer */
    int ibuf_rpos;
    /* the input-buffer itself */
    char *ibuf;
    /* write position in output-buffer */
    int obuf_wpos;
    /* read position in output-buffer */
    volatile int buf_rpos;
    char *obuf;
    struct wait_queue *skel_wait_iq;
    struct wait_queue *skel_wait_oq;
    [...]
}
#define SKEL_IBUF_EMPTY(b) \
 ((b)->ibuf_rpos==(b)->ibuf_wpos)
#define SKEL_OBUF_EMPTY(b) \
 ((b)->obuf_rpos==(b)->obuf_wpos)
#define SKEL_IBUF_FULL(b) \
 (((b)->ibuf_wpos+1)%SKEL_IBUFSIZ==(b)->ibuf_rpos)
#define SKEL_OBUF_FULL(b) \
 (((b)->obuf_wpos+1)%SKEL_OBUFSIZ==(b)->obuf_rpos)
Static int skel_open (struct inode *inode,
                      struct file *filp) {
    /* .... */
    /* First we allocate the buffers */
    board->ibuf = (char*) kmalloc (SKEL_IBUFSIZ,
                                   GFP_KERNEL);
    if (board->ibuf == NULL)
        return -ENOMEM;
    board->obuf = (char*) kmalloc (SKEL_OBUFSIZ,
                                   GFP_KERNEL);
    if (board->obuf == NULL) {
        kfree_s (board->ibuf, SKEL_IBUFSIZ);
        return -ENOMEM;
    }
    /* Now we clear them */
    ibuf_wpos = ibuf_rpos = 0;
    obuf_wpos = obuf_rpos = 0;
    board->irq = board->hwirq;
    if ((err=request_irq(board->irq>
                         skel_interrupt,
                         SA_INTERRUPT, "skel")))
        return err;
}
Static void skel_interrupt(int irq,
                    struct pt_regs *unused) {
    int i;
    Skel_Hw *board;
    for (i=0, board=skel_hw; i<skel_boards;
         board++, i++)
         /* spurious */
        if (board->irq==irq) break;
    if (i==skel_boards) return;
    if (board_is_ready_for_input)
        skel_hw_write (board);
    if (board_is_ready_for_output)
        skel_hw_read (board);
}
Static inline void skel_hw_write (Skel_Hw *board){
    int rpos;
    char c;
    while (! SKEL_OBUF_EMPTY (board) &&
        board_ready_for_writing) {
        c = board->obuf [board->obuf_rpos++];
        write_byte_c_to_device
        board->obuf_rpos %= SKEL_OBUF_SIZ;
    }
    /* Sleeping Beauty */
    wake_up_interruptible (board->skel_wait_oq);
}
Static inline void skel_hw_read (Skel_Hw *board) {
    char c;
    /* If space left in the input buffer & device ready: */
    while (! SKEL_IBUF_FULL (board) &&
        board_ready_for_reading) {
        read_byte_c_from_device
        board->ibuf [board->ibuf_wpos++] = c;
        board->ibuf_wpos %= SKEL_IBUFSIZ;
    }
    wake_up_interruptible (board->skel_wait_iq);
}
Static int skel_write (struct inode *inode,
                       struct file *file,
                       char *buf, int count) {
    int n;
    int written=0;
    Skel_Hw board =
        (Skel_Hw*) (file->private_data);
    for (;;) {
        while (written<count &&
               ! SKEL_OBUF_FULL (board)) {
            board->obuf [board->obuf_wpos] =
                get_user_byte (buf);
            buf++; board->obuf_wpos++;
            written++;
            board->obuf_wpos %= SKEL_OBUFSIZ;
        }
        if (written) return written;
        if (file->f_flags & O_NONBLOCK)
            return -EAGAIN;
        current->timeout = jiffies + SKEL_TIMEOUT;
        interruptible_sleep_on (
            &(board->skel_wait_oq));
        /* Why did we return? */
        if (current->signal & ~current->blocked)
        /* If the signal is not not being
           blocked */
            return -ERESTARTSYS;
        if (!current->timeout)
            /* no write till timout: i/o-error */
            return -EIO;
    }
}
Static int skel_read (struct inode *inode,
                      struct file *file,
                      char *buf, int count) {
    Skel_Hw board =
        (Skel_Hw*) (file->private_data);
    int bytes_read = 0;
    if (!count) return 0;
    if (SKEL_IBUF_EMPTY (board)) {
        if (file->f_flags & O_NONBLOCK)
            /* Non-blocking */
            return -EAGAIN;
        current->time_out = jiffies+SKEL_TIMEOUT;
        for (;;) {
            skel_tell_hw_we_ask_for_data
            interruptible_sleep_on (
                &(board->skel_wait_iq));
            if (current->signal
                & ~current->blocked)
                return -ERESTARTSYS;
            if (! SKEL_IBUF_EMPTY (board))
                break;
            if (!current->timeout)
                /* Got timeout: return -EIO */
                return -EIO;
        }
    }
    /* if some bytes are here, return them */
    while (! SKEL_IBUF_EMPTY (board)) {
        put_user_byte (board->ibuf
                          [board->ibuf_rpos],
                       buf);
        buf++; board->ibuf_rpos++;
        bytes_read++;
        board->ibuf_rpos %= SKEL_IBUFSIZ;
        if (--count == 0) break;
    }
    if (count) /* still looking for some bytes */
        skel_tell_hw_we_ask_for_data
    return bytes_read;
}