Memory Leak Detection in Embedded Systems

Cal discusses mtrace, dmalloc and memwatch—three easy-to-use tools that find most application program errors.
dmalloc

The third tool is a library that is designed as a drop-in substitute for malloc, realloc, calloc, free and other memory management functions. It provides runtime configurability. The features of the tool provide memory leak tracing and fencepost write detection. It reports its errors by filename and line number and logs some general statistics. This library, created and maintained by Gray Watson, has been ported to many operating systems other than Linux.

The package is configurable to include thread support and C++ support. It can be built both as shared and static libraries. All of these options are selected when building the tool. The result is a set of libraries that are used when linking the application program. There is an include file (dmalloc.h) that needs to be included in the source of the application to be checked. In addition to the library and include file, it is necessary to have an environment variable set up that dmalloc reads to configure how it checks and where it puts the logging information. The following line is the setup used with the test program for dmalloc:

export \
DMALLOC_OPTIONS=debug=0x44a40503,inter=1,log=logfile

What this means is 1) log is a file named logfile in the current directory, 2) inter is the frequency for the library to check itself and 3) debug is a hex number whose bits select the types of checking to do. This example tests for just about every possible error. The following is a list of the tests and the corresponding bits to set in “debug”:

  • none (nil): no functionality (0)

  • log-stats (lst): log general statistics (0x1)

  • log-non-free (lnf): log non-freed pointers (0x2)

  • log-known (lkn): log only known non-freed (0x4)

  • log-trans (ltr): log memory transactions (0x8)

  • log-admin (lad): log administrative info (0x20)

  • log-blocks (lbl): log blocks when heap-map (0x40)

  • log-bad-space (lbs): dump space from bad pointers (0x100)

  • log-nonfree-space (lns): dump space from non-freed pointers (0x200)

  • log-elapsed-time (let): log elapsed time for allocated pointer (0x40000)

  • log-current-time (lct): log current time for allocated pointer (0x80000)

  • check-fence (cfe): check fencepost errors (0x400)

  • check-heap (che): check heap adm structs (0x800)

  • check-lists (cli): check free lists (0x1000)

  • check-blank (cbl): check mem overwritten by alloc-blank, free-blank (0x2000)

  • check-funcs (cfu): check functions (0x4000)

  • force-linear (fli): force heap-space to be linear (0x10000)

  • catch-signals (csi): shut down program on SIGHUP, SIGINT, SIGTERM (0x20000)

  • realloc-copy (rco): copy all re-allocations (0x100000)

  • free-blank (fbl): overwrite freed memory space with BLANK_CHAR (0x200000)

  • error-abort (eab): abort immediately on error (0x400000)

  • alloc-blank (abl): overwrite newly alloced memory with BLANK_CHAR (0x800000)

  • heap-check-map (hcm): log heap-map on heap-check (0x1000000)

  • print-messages (pme): write messages to stderr (0x2000000)

  • catch-null (cnu): abort if no memory available (0x4000000)

  • never-reuse (nre): never reuse freed memory (0x8000000)

  • allow-free-null (afn): allow the frees of NULL pointers (0x20000000)

  • error-dump (edu): dump core on error and then continue (0x40000000)

If the library needs to check C++ programs, a source file named dmalloc.cc is needed with the application. This module provides wrapper functions for new to malloc and delete to free. The GNU debugger GDB can be used with dmalloc. A file is included with the product that can be used as part of a .gdbinit file so that GDB is set up automatically to know about dmalloc.

Along with the library is a utility named dmalloc that will programmatically set up the DMALLOC_OPTIONS variable. I've made a setup script that is sourced prior to running the program to be debugged. This way the setup is repeatable without errors.

This article only covers the general use of the tool, but the documentation for it is extensive and details more available features. The test program used with the earlier tools was run using DMALLOC. The results can be lengthy and optionally include the bytes present at critical areas of memory such as the “fence-top”, where a pointer overruns a malloc'd area. The end of the log file includes statistics, addresses, block sizes and line numbers for occurrences of malloc without free.

The three tools provide varying support for memory leak detection and reporting. Each of these tools has been used on a Linux workstation as well as cross-compiled and executed on several different target architectures. In one application, developers used all three tools. mtrace was used to find a memory leak in a third-party C++ library where an exception throw/catch block caused a major leak. The dmalloc tool was used to find memory leaks in the execution of Linux pthreaded applications. The memwatch tool was used to catch a buffer pool mechanism that was not properly defragmenting itself. These tools are small and easy to implement and remove when debugging is completed.

The example program consists of one source file, my_test.c. There are three separate directories that contain a README, Makefile and a script to run the test [available at ftp.linuxjournal.com/pub/lj/listings/issue101/6059.tgz]. In the dmalloc test, the environment setup script is also available. Each of the tests has been built on Red Hat and SuSE Linux releases, as well as the MontaVista Linux cross-development environments.

Resources

Cal Erickson (cal_erickson@mvista.com) currently works for MontaVista Software as a senior Linux consultant. Prior to joining MontaVista, he was a senior support engineer at Mentor Graphics Embedded Software Division. Cal has been in the computing industry for over 30 years with experience at computer manufacturers and end-user development environments.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Same memwatch log for the 2 application

Anonymous's picture

Hi,

Will the 2 applications dump the memwatch logs in the same memwatch.log?

Yes, it will append the log

Anonymous's picture

Yes, it will append the log to the existing memwatch.log file

doesn't work

gfengster's picture

The paper is good. But the link
ftp://ftp.ssc.com/pub/lj/listings/issue101/6059.tgz doesn't contain all files.
I can not prove the author is right.

Memleak

Ajit john's picture

check the memory alocated and deallocated

one more tool for debugging memory leaks-Coverity Prevent

victor louis's picture

You can prevent memory leaks by watching for some common problems. Collection classes, such as hash tables and vectors, are common places to find the cause of a memory leak. This is particularly true if the class has been declared static and exists for the life of the application. . The prevalence of memory leak bugs has led to the development of a number of debugging tools to detect unreachable memory. Coverity Prevent is one of the tool that you can use for fixing these kinds of bugs. Coverity Prevent is also used by the Department of Homeland security to scan many open source projects. You can get more info at http://www.Coverity.com

How to Prevent memory leaks

victor louis's picture

You can prevent memory leaks by watching for some common problems. Collection classes, such as hash tables and vectors, are common places to find the cause of a memory leak. This is particularly true if the class has been declared static and exists for the life of the application. . The prevalence of memory leak bugs has led to the development of a number of debugging tools to detect unreachable memory. Coverity Prevent is one of the tool that you can use for fixing these kinds of bugs. Coverity Prevent is also used by the Department of Homeland security to scan many open source projects. You can get more info at http://www.Coverity.com

why advertise the article in

Anonymous's picture

why advertise the article in an expensive book to only find that lj is nickeling and diming

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix