Callgrind slow with the equipment turned off

Question

I use callgrind to profile a multi-threaded linux application and basically it works fine. I start it with the toolkit turned off ( - instr-atstart = no ), and then, once the setup is complete, I turn it on with callgrind_control -i on . However, when I change some configurations to try to profile another part of the application, it starts to work very slowly even before turning on the toolkit. Basically, the part of the code, which takes several seconds during normal operation, takes more than an hour using callgrind (instrumentation is turned off). Any ideas as to why this might be and how to go about debugging / resolving slowness?

Answer 1

Callgrind is a tool built on valgrind. Valgrind is basically a dynamic binary translator (libVEX, part of valgrind). It will decode each instruction and JIT-compile them into a stream of some instructions of the same CPU.

As I know, there is no way to enable this translation (in the valgrind implementation) for an already running process, so the dynamic translation is turned on all the time, starting from the beginning of the program. It also cannot be disabled.

The tools are built on valgrind, adding some toolkit code. The Nul tool (nulgrind) is a tool that does not add tools. But each tool uses valgrind, and dynamic translation is active all the time. Turning on and off in callgrind is just turning on and off additional tools.

The virtual processor implemented by Valgrind is limited, there is a (incomplete) list of restrictions http://valgrind.org/docs/manual/manual-core.html#manual-core.limits . Most restrictions apply to floating point operations, and they can be emulated incorrectly.

Is the change related to floating point operations? Or with the other restrictions listed?

You should also know that "Valgrind serializes so that only one thread starts at a time." (from the same manual-core.html page)