Unreproducible runtime errors - general approach?

Question

I ran into a problem that is so mysterious that I don’t even know how to formulate this question ... I can’t even post any piece of code.

I myself am developing a large project, starting from scratch. It almost frees up time, but I can’t get rid of some annoying mistake. My program writes an output file from time to time, and during this I get either:

• std :: string error out_of_range
• std :: string length_error
• just a lot of nonsense on the way out

It is worth noting that these errors appear very rarely and can never be reproduced even with the same input. Memcheck does not detect a memory violation, even in cases where errors have been previously noted. Cppcheck also does not complain. I use STL and pthreads intensively, but without the last error also occur.

I tried both the latest g ++ and icpc. I am running some version of Ubuntu, but I don’t think that is the reason.

I would be grateful for any help from you guys in how to solve such problems. Thanks in advance.

Answer 1

Enable coredumps (ulimit -c or setrlimit ()), get the kernel and run gdb'ing. Or, if you want, do the setup in which you always work under gdb, so that when the error eventually occurs, you have information available.

Answer 2

Symptoms indicate memory corruption.

If I were to guess, I would say that something distorts the internal state of the std::string object you are writing. Is a string object on the stack? Did you remove the stack break as a possible reason (this would not be detected by valgrind )?

I also suggest running the executable file under the debugger, configured so that it will cause a breakpoint when a problem occurs. This will allow you to examine the state of your process at this point, which may be useful in determining what is happening.

Answer 3

gdb and valgrind are very useful tools for debugging such errors. valgrind especially effective in identifying memory access problems and memory leaks.

Answer 4

In rare cases, I found strange optimization errors in gcc (e.g. ++i compiled before i++ ). You can try declaring some critical volatile variables, but if valgrind doesn't find anything, the probability will be low. And of course, this is like shooting in the dark ...

If at least you can find that something is wrong in a certain run from within the program, for example, when it detects meaningless output, you could call the empty function "gotNonsense ()", with which you can break in gdb .

Answer 5

If you cannot determine exactly where in your code your program crashes, one way to find this place is to use debug output. Debugging output is a good way to debug errors that cannot be reproduced, because next time you will get more information about the error, without having to actively reproduce it. I recommend using some log files for this, like boost.

Answer 6

You are heavily using STL, so you can try running the program using libstdc ++ c. It will perform additional checks on iterators, containers, and algorithms. To use libstdC ++ debugging mode, compile the application using the compiler flag -D_GLIBCXX_DEBUG