Loadable Kernel Module Programming and System Call Interception

Loadable Kernel Module Programming and System Call Interception
3. A More Interesting Example: Intercepting sys_execve to Protect Against


After a machine is compromised, malicious users tend to replace commonly used programs with trojan horses (programs that execute malicious instructions in addition to their normal functions). Packages of such trojan horses are widely distributed over the Internet and are easily accessible by anyone. Therefore, it becomes important to protect programs from being replaced by malicious users.

In order to protect against such problems, our next example involves the interception of various system calls, most importantly sys_execve, to check the hash of the program to be executed against a known hash present in a database file. The program is denied execution if the hashes do not match, and such an attempt is logged. One way to implement this is seen in the following steps:

  1. Intercept sys_execve and compute the inode of the file being executed, than compare it with the inodes of the files present in the hash database. Inodes are data structures that contain information about files in the file system. Since there is a unique inode for each file, we can be certain of our comparison results. If no match is found, call the original sys_execve and return. However, if a match is found, compute the hash of the program and then compare it with the hash present in the hash database. If they match, call the original sys_execve and return. If they do not match, log the attempt and return an error.

  2. Intercept sys_delete_module. If called with our module name as the parameter, return an error. Our module cannot be deleted.

  3. Intercept sys_create_module, and return an error. No more modules can be inserted because we do not want any malicious module to be able to intercept the sys_execve described in step 1.

  4. Intercept sys_open to prevent our hash database and log file to be opened for writing.

  5. Intercept sys_unlink to prevent deletion of the hash database and log file.

Note that the above does not offer complete protection, but it is a simple first-step implementation. For example, a malicious user may modify kernel symbols in /dev/kmem or use raw device access to the hard disk, and bypass open to write to the hash database file. Also, since our implementation is only a loadable module, a malicious user can alter our /etc/rc.d files and stop our module from being loaded the next time the machine is rebooted. In addition, various other system calls exist that could cause our hash database and log files to be altered or deleted.

At this time, it becomes important to acknowledge the possibility of loadable module support being misused by a malicious user. For example, the sys_execve function call can be intercepted to invoke a trojan program instead of the one intended, and system calls such as read and write can be intercepted to perform keystroke logging. Therefore, the flexibility and power of loadable kernel modules can be misused by malicious users who may have gained access to the system. See Resources for a web site that has details of this example along with complete source code.



Nitesh Dhanjani is a graduate student at Purdue University. His interests are operating systems, networking and security. He has performed security audits and reviews for various firms including Ernst & Young LLP, and offers consulting services in his spare time. He can be reached at dhanjani@dhanjani.com.

Gustavo Rodriguez-Rivera is a Visiting Assistant Professor at Purdue University and is also software architect for Geodesic Systems. His interests are operating systems, networking and memory management. He can be reached at grr@cs.purdue.edu.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Compile error

Anonymous's picture

intercept.c:2:26: error: linux/module.h: No such file or directory
intercept.c:8: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
intercept.c:10: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'int'
intercept.c: In function 'init_module':
intercept.c:26: error: 'original_sys_exit' undeclared (first use in this function)
intercept.c:26: error: (Each undeclared identifier is reported only once
intercept.c:26: error: for each function it appears in.)
intercept.c:31: error: 'our_fake_exit_function' undeclared (first use in this function)
intercept.c: In function 'cleanup_module':
intercept.c:43: error: 'original_sys_exit' undeclared (first use in this function)

As you can see, linux/module.h cannot be included, although it exists in /usr/include.
Any help, please?

Broken links

Anonymous's picture

On the resources page, the link to the second example (intercepting execve) is broken. Is there somewhere else to obtain the code for the second example?

This article is referenced as a tutorial on Stack Overflow, here:


... a lot of people will be wanting a peek at the code for the second example :)

Actually, all links are broken

Anonymous's picture

All links on the resources / reference page appear to now be broken (not surprising considering the date of this article)

Some problem

iPAS's picture

I compiled your example2.c then found this...

> /usr/include/linux/kernel.h:72: error: syntax error before "size_t"
> ...

Could you tell me about kernel version & gcc version that you used ?

One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix