Geek Answers Handbook

How can I profile a multithreaded program?

I have a program that runs waaaay under the par parameter, and I would like to profile it. However, this is multithreading, so I cannot find a good way to profile this thing. Any advice?

I tried yappi, but it segfaults on OS X :(

EDIT: this is in python, sorry for being under profiling ...

+4
source share
2 answers

Multithreading or multiprocessing? If you're just multithreading, then this is a problem. Python is currently having problems with multi-threaded processing in a multi-processor system due to Global Interpreter Lock (GIL). They are working on fixing it for Python 3.2 - at least so that your program runs as fast on one core as it does on multiple kernels.

If you're not sure, see the results of the shootout for the thread program. Working with one core is faster than working with quad core .

Now, if you use multiprocessing, profiling can be difficult, because then you need to run CProfiler from each individual process. There are several questions that point you in the right direction.

+2
source

Depending on how far you find your troubleshooting solution, there are some tools that can point you in the right direction.

  • "top" is a good start to show you if your problem is burning CPU time or just waiting for material.

  • "dtruss -c" can show you where you spend time and which system calls take up most of your time.

Both of them can give you a hint without knowing anything about python.

If you just want to use yappi, don't work too hard to set up virtual boxing and install some kind of Linux on your computer. I find that I do this from time to time when I want to try something.

Of course, there may be things that I don’t know about, which makes it impossible or not worth the effort. In addition, profiling on another operating system running virtualization may not produce exactly the same results, but it may be useful.

+2
source

More articles:


All Articles