What do i need to debug pthreads?

Question

I want to debug pthreads in my custom Linux distribution, but I'm missing something. My host is Ubuntu 12.04, my goal is embedded Linux i486, built into the crosstool-NG cross-compiler suite, the rest of the OS is done using Buildroot.

I will list the facts:

• I can run multithreaded applications for my target

• Google Breakpad is unable to generate a crash report when running a multithreaded application on the target server. The exact same application with the same Breakpad library build will succeed when I run it on my host.

• GDB cannot debug multithreaded applications for my purpose.

eg.

$./gdb -n -ex "thread apply all backtrace" ./a.out --pid 716 dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs GDB will not be able to debug pthreads. GNU gdb 6.8 

I do not think ps_lgetfpregs is a problem because of this .

• My crosstool assembly created the libthread_db.so file and I put it on target.

• My crosstool assembly created gdb for my purpose, so it should have been linked to the same libraries that I run on the target.

• If I run gdb on my host, against my test application, I get a back trace of every working thread.

I suspect that the problem with Breakpad is related to the GDB problem, but I can not substantiate this. The only common thing is the lack of multi-threaded debugging.

There is some important difference between my host and the target, which prevents me from debugging pthreads on the target.

Does anyone know what it is?

EDIT:

Denis Dmitrienko from TI says:

GDB is usually not very picky, and you can mix and match different versions of gdb and gdbserver. But unfortunately, if you need to debug multi-threaded applications, there are some dependencies for specific APIs, ...

For example, this is one of the messages that you can see if you do not build GDB correctly to support threads:

dlopen failed to execute 'libthread_db.so.1' - / lib / libthread_db.so.1: undefined symbol: ps_lgetfpregs GDB will not be able to debug Pthreads.

Please note that this error is the same as the one I get, but it does not understand how to build GDB correctly.

and GDB Questions :

(Q) GDB does not see any stream except the one in which the failure occurred; or SIGTRAP kills my program when I set a breakpoint.

(A) This often happens on Linux, especially for embedded purposes. There are two common reasons:

• you use glibc and you split libpthread.so.0

• mismatch between libpthread.so.0 and libthread_db.so.1

GDB itself does not know how to decode the "flow control blocks" supported by glibc and is considered as a private detail of the glibc implementation. It uses libthread_db.so.1 (part of glibc) to help him do this. Therefore, libthread_db.so.1 and libpthread.so.0 must match in version and compilation flags. In addition, libthread_db.so.1 requires certain non-global characters that will be present in libpthread.so.0.

Solution: use strip --strip-debug libpthread.so.0 instead of strip libpthread.so.0.

I tried not without libpthread.so.0, but that did not help. I will investigate any discrepancy between pthread and thread_db.

Answer 1

It:

 dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_lgetfpregs GDB will not be able to debug pthreads. 

means libthread_db.so.1 library could not find ps_lgetfpregs symbol in gdb.

Why?

Since I built gdb using Crosstoolg-NG with the option "Build a static native gdb", and this adds the -static option to gcc.

Built-in gdb is built with the -rdynamic option, and this populates the character table . dynsym in an ELF file with all characters, even unused. libread_db uses this symbol table to find ps_lgetfpregs from gdb.

But -static removes the .dynsym table from the ELF file.

There are two options at this point:

• Do not create static native gdb if you want to debug threads.
• Create static gdb and static libthread_db (not tested)

Edit:

By the way, this does not explain why Breakpad cannot debug multithreaded applications for my purpose.

Answer 2

Simple though ... To use the gdb debugger, you need to compile your code with the -g option. For example, gcc -g -c * .c.