Writing a Path Covering Tool

Question

One of our production systems currently has over 3,000 programs written between 1986 and now. The code base is written in a non-standard language, unfortunately, there are not enough modern testing tools.

To improve the quality of the code, I am working on the implementation of processes and the creation of tools that will improve development and testing. I only have a line coverage tool so that we can help identify dead code + unverified code during development.

Now I would like to start work on adding Path Coverage to the tool.

How can I do it?

Given that:

1) The line coverage tool acts as a pre-processor that enters the code
2) I already have the opportunity to collect statistics installed in the specified code.

What data should be recorded as the program runs and how to interpret it?

How can I present the results via HTML?

I already read the question. How to start writing a code coverage tool? , which was about Java, but it didn’t help (including the paper "Industry coverage for arbitrary languages ​​made simple").

Thanks in advance for any recommendations!

Answer 1

To cover the route, you need to somehow get into the program control flow. The obvious method is to build a real control flow graph and then intersect its segments to highlight “path fragments” (for example, base paths) that will be used in your path coverage analysis. (You can try to do this by hacking the source code, but you are likely to fail; parsing and parsing is too complicated).

See What is the coverage point of the base path? for a good discussion of stackoverflow on basic paths.

To implement the necessary tool to cover the path, you probably need to fully analyze the full inherited language. For 3000 programs and strong testing requirements, the use of an industrial capacity analyzer and infrastructure for this makes sense.

Our DMS Software Reengineering Toolkit can be used to build not only an analyzer, but also an analysis of the control flow and tools needed to collect coverage data paths. ("Industry coverage for arbitrary languages" did this if all you wanted to do was collect branch coverage data, but more for DMS than just parsing). DMS also supports the construction of control schedules (and data flow) if you need them, as you do in this case; see DMS built control flow graphs .

DMS is used to create parsers for analyzing and analyzing data flow for C, Java and COBOL, and was used to create parsers for about 30 + langauges. It can handle your outdated language if you are serious about it.

EDIT 10/31/2011: DMS can now calculate the control flow for C ++, so it will be an excellent basis for a C ++ path coverage tool.

Answer 2

Measuring track coverage is a complex issue. You must first determine what you mean by the path in the first place. Is a loop executed three times a different way than a loop executed four times? If so, you have an infinite number of paths. If not, there are test cases, even if all paths are covered.

Perhaps the best next step is branch coverage: measuring whether each condition is true or false. This can be done by recording sequences of line numbers.