klogd: The Kernel Logging Dæmon

klogd reads kernel log messages and helps process and send those messages to the appropriate files, sockets or users. This month we discuss memory address resolution and how to modify klogd's default behavior using command-line switches.
Multiple Kernels

Linux beginners are unlikely to encounter this, but more seasoned users will often have more than one bootable kernel on their system at a time. If the box is a hobbyist or kernel hacker's box, it is likely to have a number of stable and a number of development series kernels on it. I myself always keep three generations of stable kernels on my systems, so that if a bug should show up, I can immediately reboot into the older kernel.

When klogd starts, it identifies the kernel version (all kernels since 1.3.43 put version information in the map files) and then looks at:

/boot/System.map
/System.map
/usr/src/linux/System.map

It will use the kernel version information to choose the correct one, if possible. When I have a development series kernel on a box, I leave the stable kernel map in /boot and I leave the development kernel map in /usr/src/linux. As for my two “old” stable kernels, I just live with the fact that klogd will not be able to resolve addresses in the event of a fault if I boot into them. Remember, you can use the -k switch on klogd to force it to use a particular map file if you build an archive of them.

Just bear in mind as you read this discussion that these events are so rare that in 15 machine years (three machines running Linux 24x7 for five years), I have seen this happen twice, and both times it was due to failing hardware.

Signals

In addition to the command-line switches, klogd will respond to certain signals. You send signals with the kill command.

The signals klogd responds to are:

  • SIGTSTP/SIGCONT

SIGTSTP suspends and SIGCONT resumes kernel logging. The resuming includes re-initialization, so you can use this to, for example, unmount the /proc file system without killing klogd:

kill -TSTP <pid>
umount /proc
kill -CONT <pid>
  • SIGUSR1/SIGUSR2

SIGUSR1 causes a reload of the kernel module symbols. SIGUSR2 reloads both the static kernel symbols and the module symbols.

See the Memory Address Resolution section for more information.

  • SIGINT/SIGHUP/SIGKILL/SIGTERM

These signals all gracefully shut down klogd.

Summary

The klogd works with syslogd to handle the dispatch of kernel messages. It exists solely because the kernel itself is unable to use the syslog API directly. Klogd provides for resolving raw memory addresses into kernel symbol names.

Michael A. Schwarz (mschwarz@sherbtel.net) is a consultant with Interim Technology Consulting in Minneapolis, Minnesota. He has 15 years of experience writing UNIX software and heads up the open-source SASi project. He has been using Linux since he downloaded the TAMU release in 1994, and keeps the SASi project at http://alienmystery.planetmercury.net/.

______________________

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix