Remote Debugging of Loadable Kernel Modules with kgdb: a Knowledge-based Article for Getting Started
We are almost ready to begin debugging a module. But before we proceed, we have to alleviate another possible issue. In gdb we will be using the add-symbol-file to inform the debugger of the memory location for our module's object code on the target machine. However, gdb 5.0 and previous versions have had problems in correctly calculating addresses using the add-symbol-file command (the problem surrounds the issue of a module's global variables). The problem has been corrected in developmental versions of gdb. It is therefore recommended that you use a developmental version of gdb to debug modules with kgdb. The gdb version used for the remainder of this article is a developmental gdb built for Red Hat 6.2. For more information regarding this issue, visit http://kgdb.sourceforge.net/.
The last step we must account for is the configuration of our module to communicate with the remote gdb session. This is a relatively simple task. We now know that when gdb on the development machine makes the initial contact with the target machine, the gdbserial interface issues a call to the debugging stub's set_debug_traps( ) function. This function, as you recall, instructs gdb to perform all exception handling for the target kernel. The serial interface then issues a call to the stub's breakpoint( ) function, which turns control over to gdb. At this point we can inform the remote kernel to resume normal operations by issuing a continue command from gdb.
With the target kernel now configured to return control to the remote gdb session whenever an exception is triggered, we can modify our module code to guarantee that such an event will arise. Consider the following modifications made to the simple module in Listing 6.
As you can see by the code in Listing 6, we have added a breakpoint interrupt that is to be called (in this example) when the module is both loaded and unloaded from the kernel. These interrupts will return control to the remote gdb session, thus halting execution of the module at those points. Let's try it out.
On the development machine, recompile the module after adding the BREAKPOINT code:
gcc -c -O2 -g simple.c
Copy the newly compiled object code to the target machine.
Initiate a remote debug session on the target machine by running the gdbstart program.
On the development machine, navigate to /usr/src/linux and run gdb vmlinux. Remember to use a developmental version of the debugger as described in the previous section.
Once prompted in gdb, type rmt to initiate contact with the remote machine.
With the previously recorded hex address, use add-symbol-file to instruct gdb of the modules location in memory:
add-symbol-file /root/simple.o 0xc480004c
You may be wondering at this point how we can assign this address space when the module is not currently loaded. Or a better question may be, ``How do we know that the kernel will load the module into the exact same location in memory?'' We can make this assumption because the kernel very often will load the object code into the same memory segment as before. While this is not written in stone it does happen frequently enough to render this method quite reliable.
Return control to the remote kernel by issuing a continue command from gdb.
On the target machine, install the module:
insmod simple.oWhen insmod invokes the init_module modules, our breakpoint interrupt is called and returns control to gdb. This allows us to step through the remainder of the init_module as if it were a user-space application as shown in Listing 7.
Note that this sequence reflects an example of stopping the module's load process. The module could be installed with the insmod -m parameter again to verify memory placement of the object code if other module functionality (other than the initialization process) was to be debugged (e.g., file operation functions: open, read, write, close, ioctl; driver resource allocation/deallocation, etc.).
Return control to the remote kernel from gdb (i.e., continue).
On the target machine, remove the simple module:
This of course issues the cleanup_module function, which in turn invokes another of our breakpoints, returning control to gdb on the development machine:
(gdb) c Continuing. Program received signal SIGTRAP, Trace/breakpoint trap. 0xc4800068 in cleanup_module () at simple.c:38 38 } // end function init_module (gdb)
Return control to the remote kernel and stop the debug session.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Profiles and RC Files
- Understanding Ceph and Its Place in the Market
- Astronomy for KDE
- Git 2.9 Released
- Maru OS Brings Debian to Your Phone
- OpenSwitch Finds a New Home
- What's Our Next Fight?
- The Giant Zero, Part 0.x
- SoftMaker FreeOffice
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide