Loadable Kernel Module Programming and System Call Interception

Loadable Kernel Module Programming and System Call Interception

Modern CPUs can run in two modes: kernel mode and user mode. When a CPU runs in kernel mode, an extended set of instructions is allowed, as is free access to anywhere in memory and device registers. Interrupt drivers and operating system services run in kernel mode. In contrast, when the CPU runs in user mode, only a restricted set of instructions is allowed, and the CPU has a restricted view of the memory and devices. Library functions and user programs run in user mode. Kernel and user mode together form the basis for security and reliability in modern operating systems.

Programs spend most of their time in user mode and switch to kernel mode only when they need an operating system service. Operating system services are offered through system calls. System calls are “gates” into the kernel implemented with software interrupts. Software interrupts are interrupts produced by a program and processed in kernel mode by the operating system.

The operating system maintains a “system call table” that has pointers to the functions that implement the system calls inside the kernel. From the program's point of view, this list of system calls provides a well-defined interface to the operating system services. You can obtain a list of the different system calls by looking at the file /usr/include/sys/syscall.h. In Linux, this file includes the file /usr/include/bits/syscall.h.

Loadable modules are pieces of code that can be loaded and unloaded into the kernel on demand. Loadable modules add extra functionality to the kernel without the need of rebooting the machine. For example, it is common in Linux to use loadable modules for new device drivers. The alternative to loadable modules is a monolithic kernel where new functionality is added directly into the kernel code. Monolithic kernels have the disadvantage of needing to be rebuilt and reinstalled every time new functionality is added.

Kernel programming can be difficult not only because of the intrinsic complexity but also because of the long debugging cycle. Debugging an operating system may require installing a new kernel and rebooting the machine in every cycle. We strongly recommend using loadable modules in kernel development because a) there is no need to rebuild the kernel or to reboot the machine more often than necessary; and b) since the end user does not need to replace/rebuild the existing kernel, the user is more likely to install the new functionality.

Loadable module support within the Linux kernel facilitates the interception of system calls, and this feature can be taken advantage of as described within the examples below. As a note, it is assumed that the reader is familiar with C programming.

1. System Calls, an Introduction

Operating systems provide entry points through system calls that allow user-level processes to request services from the kernel. It is important to distinguish between system calls and library functions. Library functions are linked to the program and tend to be more portable since they are not bound to the kernel implementation. However, many library functions use system calls to perform various tasks within the system kernel. To illustrate, consider this C program that opens a file and prints its contents:

#include <stdio.h>
int main(void)
{
    FILE *myfile;
    char tempstring[1024];
    if(!(myfile=fopen("/etc/passwd","r")))
    {
         fprintf(stderr,"Could not open file\n");
         exit(1);
    }
    while(!feof(myfile))
    {
         fscanf(myfile,"%s\n",tempstring);
         fprintf(stdout,"%s\n",tempstring);
    }
    exit(0);
}

Within the program, we used the fopen function call in order to open the /etc/passwd file. However, it is important to note that fopen is not a system call. In fact, fopen calls the system call open internally in order to do the real I/O. To get a list of all the system calls invoked by a program, use the strace program. Assuming you have compiled the above program as a.out by running gcc example1.c, running strace like : strace ./a.out will allow you to see all the system calls being invoked by a.out.

The kernel switches to the user-id of the process owner invoking the system call. So, if a regular user were to run the above program, with /etc/shadow (which is not readable) as the parameter to fopen, the open would fail and so would fopen, causing the if clause above to translate to true, thus printing the Could not open file error message.

2. Intercepting System Calls via Loadable Modules, an Example

Assume that we want to intercept the exit system call and print a message on the console when any process invokes it. In order to do this, we have to write our own fake exit system call, then make the kernel call our fake exit function instead of the original exit call. At the end of our fake exit call, we can invoke the original exit call. In order to do this, we must manipulate the system call table array (sys_call_table). Take a look at /usr/src/linux/arch/i386/kernel/entry.S (assuming you are on an i386 architecture). This file contains a list of all the system calls implemented within the kernel and their position within the sys_call_table array.

Armed with the sys_call_table array, we can manipulate it to make the sys_exit entry point to our new fake exit call. We must store a pointer to the original sys_exit call and call it when we are done printing our message to the console. Source code to implement the above is as shown in Listing 1.

Listing 1. Example 2.c

Compile the program shown in Listing 1 by invoking gcc: gcc -Wall -DMODULE -D__KERNEL__ -DLINUX -c example2.c. This gives us our example2.o module. In order to insert this module into the kernel, do this as root: insmod example2.o. Now, make sure you are on the console (since printk only prints to the console), and run any program which uses the exit system call. For example, ls should print: HEY! sys_exit called with error_code=0.

Next, try to invoke ls with a file that does not exist; this should cause ls to call the exit system call with an argument other than 0. Therefore, ls somefilethatdoesnotexist should print: HEY! sys_exit called with error_code=1.

In order to list all the modules loaded, use lsmod. To remove the module, run rmmod example2.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Compile error

Anonymous's picture

intercept.c:2:26: error: linux/module.h: No such file or directory
intercept.c:8: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
intercept.c:10: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
intercept.c: In function 'init_module':
intercept.c:26: error: 'original_sys_exit' undeclared (first use in this function)
intercept.c:26: error: (Each undeclared identifier is reported only once
intercept.c:26: error: for each function it appears in.)
intercept.c:31: error: 'our_fake_exit_function' undeclared (first use in this function)
intercept.c: In function 'cleanup_module':
intercept.c:43: error: 'original_sys_exit' undeclared (first use in this function)

As you can see, linux/module.h cannot be included, although it exists in /usr/include.
Any help, please?

Broken links

Anonymous's picture

On the resources page, the link to the second example (intercepting execve) is broken. Is there somewhere else to obtain the code for the second example?

This article is referenced as a tutorial on Stack Overflow, here:

http://stackoverflow.com/questions/2394985/linux-kernel-add-system-call-...

... a lot of people will be wanting a peek at the code for the second example :)

Actually, all links are broken

Anonymous's picture

All links on the resources / reference page appear to now be broken (not surprising considering the date of this article)

Some problem

iPAS's picture

I compiled your example2.c then found this...

> /usr/include/linux/kernel.h:72: error: syntax error before "size_t"
> ...

Could you tell me about kernel version & gcc version that you used ?

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState