Stack Backtracing Inside Your Program

How to use a backtrace to follow the execution path and find out what went wrong and where.
Hazards and Limitations

A couple of points are important to keep in mind when you use the backtrace functions. First, backtrace_symbols() internally calls malloc() and, thus, can fail if the memory heap is corrupted--which might be the case if you are dealing with a fault signal handler. If you need to resolve the return addresses in such a situation, calling backtrace_symbols_fd() is safer, because it directly writes to the given file descriptor without allocating memory. The same reasoning implies that it is safer to use either static or automatic (non dynamic) storage space for the array passed to backtrace().

Also, there are some limitations to the ability of automatically tracing back the execution of a program. The most relevant are some compiler optimizations that, in one way or another, alter the contents of the stack frame or even prevent a function from having one (think of function inlining). Obviously, the stack frame does not even exist for macros, which are not function calls at all. Finally, a stack backtrace is impossible to perform if the stack itself has been corrupted by a memory trash.

Regarding symbol resolution, the current glibc (version 2.3.1 at the time of this writing) allows users to obtain the function name and offset only on systems based on the ELF binary format. Furthermore, static symbols' names cannot be resolved internally, because they cannot be accessed by the dynamic linking facilities. In this case, the external command addr2line can be used instead.

Inner Workings

In case you wonder how would you access stack information in a C program, the answer is simple: you can't. Stack handling, in fact, depends heavily on the platform your program runs on, and the C language does not provide any means to do it in a standard way. The implementation of backtrace() in the glibc library contains platform-specific code for each platform, which is based either on GCC internal variables (__builtin_frame_address and __builtin_return_address) or on assembly code.

In the case of the i386 platform (in glibc-x.x.x/sysdeps/i386/backtrace.c), a couple of lines of assembly code are used to access the contents of the ebp and esp CPU registers, which hold the address of the current stack frame and of the stack pointer for any given function:

register void *ebp __asm__ ("ebp");
register void *esp __asm__ ("esp");

Starting from the value of ebp, it is easy to follow the chain of pointers and move up to the initial stack frame. In this way you gather the sequence of return addresses and build the backtrace.

At this point, you still have to resolve the return addresses into function names, an operation dependent on the binary format you are using. In the case of ELF, it is performed by using a dynamic linker internal function (_dl_addr(), see glibc-x.x.x/sysdeps/generic/elf/backtracesyms.c).

Conclusion

Are you working on a complex program that contains a lot of different execution paths that make you cluelessly wander through hundreds of functions, desperately trying to understand which one called which other function? Wander no more and print a backtrace. It's free, fast and easy. While you are at it, do yourself a favour and also use that function inside a fault signal handler--it's guaranteed to help you with those nasty bugs that appear once in a thousand runs.

Gianluca Insolvibile has been a Linux enthusiast since kernel 0.99pl4. He currently deals with networking and digital video research and development.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Hi, Excellent article. Is

Anonymous's picture

Hi,
Excellent article. Is there a way to resolve addresses to symbols when the executable is statically linked. It should be possible by reading through elf section i guess. Any pointers pls ?

m68k hacker

Anonymous's picture

on linux it works well, like all other stuff.
better tell me how to do this on m68k, guru!

kernel function like backtrace

Zhengju Sha's picture

Hi,
The function backtrace() is a user space function, does there exist kernel function like that?

I'm modifying some kernel modules(.ko), but it still have bugs :-( .After reading this, I want to
print my stack frame. But I don't know how. Looking forward to your advice.

Thanks.

lineno is off by one

Vinayak Datar's picture

This is helpful.

However, after passing the address to addr2line, it give line no of the next line of the calling statement.

So, in the same example, it gives does not give line no of "func_high()". Instead, it give line no of "return 2*p1;"

Any ideas to get correct line no as well?

Using 3rd arg in sigcontext doesn't work on ARM

MMysore's picture

Thanks for your article. V. helpful indeed.

I'm trying to get this method to work on ARM running embedded linux (running glibc-2.3.2... I am trying to get the PC/EIP from typecasting the 3rd argument of the signal handler as ucontext_t and finding the appropriate arch-dependent register. However, in ARM none of the registers seem to contain anything that looks like the program counter (not even R15, which should actually be the PC). So, I'm unable to find out where the exact crash happened. Any thoughts/ideas?

Thanks,
M.

backtrace_symbols for ARM

Sharan's picture

I appriciate the great work that people have put up on this page.I am having a serious problem with getting the call stack information on ARM. I have tried same kind of code, which works fine for linux on x386 with -rdynamic option of gcc but fails on ARM. It would be great if you could help me on this.

Are there any specific options for compiling this on ARM,if YES then what is that option?.

Thanks in advance,

Sharan.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

My problem is to backtrace on a powerPC based Linux platform.
Where can I find address of interrupted program?
powerPC has no eip, nor program counter at all ...
Thank you.

I have tried this and it work

Anonymous's picture

I have tried this and it works great, but now I have no core files generated. Any ideas on how to fix?
Thanks.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

on linux it works well, like all other stuff.
better tell me how to do this on tru64, guru!

Re: Stack Backtracing Inside Your Program

Anonymous's picture

backtrace is neat when it works.

It still lacks many features you get for free in gdb,
or in the Win32 StackWalk facilities:

. It can't seem to properly convert addresses to names
when dealing with static functions.

. It doesn't demangle C++ names, which obliges to
call the obscure and ill-documented __cxa_demangle

. It can't get you your local variables

. It can't get you source references (file+line no)

Re: Stack Backtracing Inside Your Program

Anonymous's picture

main()
{
printf(get the value of i=1,i<=5,i=1)

what will be reasulte

Re: Stack Backtracing Inside Your Program

Anonymous's picture

main()
{
printf(get the value of i=1,i<=5,i=1)

what will be reasulte

interested

Anonymous's picture

same question here. really wondering if you can answer that, guru?

What about locals -vs- arguments?

Anonymous's picture

Is there any easy way to display the rest of the local frame? I'd love to be able to get at the parameters passed. I could just dump the memory added to the stack before the address, but I don't know how far to go.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

You can access the stack in C, but it is still true that some platform-specific knowledge is needed to make sense of it. Just take the address of an argument or automatic variable and work from there:
void func( int arg )
{
void *stackframe = &arg;
...
}

Stack arguments ot a backtrace

Otto Wyss's picture

I'd appreciate if you could detail the stack argument access a little better. I'm written a wxCrashPrint component for the wxWidgets (wxWindows) framework (here) and would like to show arguments as well.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

Very cool!

I can see where this could help in situations where bugs disappear when debug mode is turned on.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

printf in a signal handler? All y'all are just asking for trouble in the future.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix