Stack Backtracing Inside Your Program

How to use a backtrace to follow the execution path and find out what went wrong and where.
Hazards and Limitations

A couple of points are important to keep in mind when you use the backtrace functions. First, backtrace_symbols() internally calls malloc() and, thus, can fail if the memory heap is corrupted--which might be the case if you are dealing with a fault signal handler. If you need to resolve the return addresses in such a situation, calling backtrace_symbols_fd() is safer, because it directly writes to the given file descriptor without allocating memory. The same reasoning implies that it is safer to use either static or automatic (non dynamic) storage space for the array passed to backtrace().

Also, there are some limitations to the ability of automatically tracing back the execution of a program. The most relevant are some compiler optimizations that, in one way or another, alter the contents of the stack frame or even prevent a function from having one (think of function inlining). Obviously, the stack frame does not even exist for macros, which are not function calls at all. Finally, a stack backtrace is impossible to perform if the stack itself has been corrupted by a memory trash.

Regarding symbol resolution, the current glibc (version 2.3.1 at the time of this writing) allows users to obtain the function name and offset only on systems based on the ELF binary format. Furthermore, static symbols' names cannot be resolved internally, because they cannot be accessed by the dynamic linking facilities. In this case, the external command addr2line can be used instead.

Inner Workings

In case you wonder how would you access stack information in a C program, the answer is simple: you can't. Stack handling, in fact, depends heavily on the platform your program runs on, and the C language does not provide any means to do it in a standard way. The implementation of backtrace() in the glibc library contains platform-specific code for each platform, which is based either on GCC internal variables (__builtin_frame_address and __builtin_return_address) or on assembly code.

In the case of the i386 platform (in glibc-x.x.x/sysdeps/i386/backtrace.c), a couple of lines of assembly code are used to access the contents of the ebp and esp CPU registers, which hold the address of the current stack frame and of the stack pointer for any given function:

register void *ebp __asm__ ("ebp");
register void *esp __asm__ ("esp");

Starting from the value of ebp, it is easy to follow the chain of pointers and move up to the initial stack frame. In this way you gather the sequence of return addresses and build the backtrace.

At this point, you still have to resolve the return addresses into function names, an operation dependent on the binary format you are using. In the case of ELF, it is performed by using a dynamic linker internal function (_dl_addr(), see glibc-x.x.x/sysdeps/generic/elf/backtracesyms.c).

Conclusion

Are you working on a complex program that contains a lot of different execution paths that make you cluelessly wander through hundreds of functions, desperately trying to understand which one called which other function? Wander no more and print a backtrace. It's free, fast and easy. While you are at it, do yourself a favour and also use that function inside a fault signal handler--it's guaranteed to help you with those nasty bugs that appear once in a thousand runs.

Gianluca Insolvibile has been a Linux enthusiast since kernel 0.99pl4. He currently deals with networking and digital video research and development.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Hi, Excellent article. Is

Anonymous's picture

Hi,
Excellent article. Is there a way to resolve addresses to symbols when the executable is statically linked. It should be possible by reading through elf section i guess. Any pointers pls ?

m68k hacker

Anonymous's picture

on linux it works well, like all other stuff.
better tell me how to do this on m68k, guru!

kernel function like backtrace

Zhengju Sha's picture

Hi,
The function backtrace() is a user space function, does there exist kernel function like that?

I'm modifying some kernel modules(.ko), but it still have bugs :-( .After reading this, I want to
print my stack frame. But I don't know how. Looking forward to your advice.

Thanks.

lineno is off by one

Vinayak Datar's picture

This is helpful.

However, after passing the address to addr2line, it give line no of the next line of the calling statement.

So, in the same example, it gives does not give line no of "func_high()". Instead, it give line no of "return 2*p1;"

Any ideas to get correct line no as well?

Using 3rd arg in sigcontext doesn't work on ARM

MMysore's picture

Thanks for your article. V. helpful indeed.

I'm trying to get this method to work on ARM running embedded linux (running glibc-2.3.2... I am trying to get the PC/EIP from typecasting the 3rd argument of the signal handler as ucontext_t and finding the appropriate arch-dependent register. However, in ARM none of the registers seem to contain anything that looks like the program counter (not even R15, which should actually be the PC). So, I'm unable to find out where the exact crash happened. Any thoughts/ideas?

Thanks,
M.

backtrace_symbols for ARM

Sharan's picture

I appriciate the great work that people have put up on this page.I am having a serious problem with getting the call stack information on ARM. I have tried same kind of code, which works fine for linux on x386 with -rdynamic option of gcc but fails on ARM. It would be great if you could help me on this.

Are there any specific options for compiling this on ARM,if YES then what is that option?.

Thanks in advance,

Sharan.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

My problem is to backtrace on a powerPC based Linux platform.
Where can I find address of interrupted program?
powerPC has no eip, nor program counter at all ...
Thank you.

I have tried this and it work

Anonymous's picture

I have tried this and it works great, but now I have no core files generated. Any ideas on how to fix?
Thanks.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

on linux it works well, like all other stuff.
better tell me how to do this on tru64, guru!

Re: Stack Backtracing Inside Your Program

Anonymous's picture

backtrace is neat when it works.

It still lacks many features you get for free in gdb,
or in the Win32 StackWalk facilities:

. It can't seem to properly convert addresses to names
when dealing with static functions.

. It doesn't demangle C++ names, which obliges to
call the obscure and ill-documented __cxa_demangle

. It can't get you your local variables

. It can't get you source references (file+line no)

Re: Stack Backtracing Inside Your Program

Anonymous's picture

main()
{
printf(get the value of i=1,i<=5,i=1)

what will be reasulte

Re: Stack Backtracing Inside Your Program

Anonymous's picture

main()
{
printf(get the value of i=1,i<=5,i=1)

what will be reasulte

interested

Anonymous's picture

same question here. really wondering if you can answer that, guru?

What about locals -vs- arguments?

Anonymous's picture

Is there any easy way to display the rest of the local frame? I'd love to be able to get at the parameters passed. I could just dump the memory added to the stack before the address, but I don't know how far to go.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

You can access the stack in C, but it is still true that some platform-specific knowledge is needed to make sense of it. Just take the address of an argument or automatic variable and work from there:
void func( int arg )
{
void *stackframe = &arg;
...
}

Stack arguments ot a backtrace

Otto Wyss's picture

I'd appreciate if you could detail the stack argument access a little better. I'm written a wxCrashPrint component for the wxWidgets (wxWindows) framework (here) and would like to show arguments as well.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

Very cool!

I can see where this could help in situations where bugs disappear when debug mode is turned on.

Re: Stack Backtracing Inside Your Program

Anonymous's picture

printf in a signal handler? All y'all are just asking for trouble in the future.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState