Is mock framework and high test coverage important?

Question

Layout frames, for example. EasyMock, simplify plugin binding. Having said that, using them to ensure that the various methods for specific components are called (and in what order) seems bad to me. It provides behavior for class testing, which makes it difficult to maintain production code. And I really don't see the benefits; mentally I feel that I was chained to a heavy ball.

I really wanted to just test the interface by providing test data as input and asserting the result. Even better, use some testing tool that automatically generates test data to validate this property. for example, adding one item to a list and deleting it immediately leads to the same list.

At our workplace, we use Hudson, which provides testing. Unfortunately, this makes it easy to dazzle that everything is tested. I strongly feel that you do not need to check everything if you want to be productive and in maintenance mode. One good example would be controllers in web frameworks. As a rule, they should contain very little logic, testing with the help of fake frameworks that the controller calls such and such a method in a certain order is meaningless in my honest opinion.

Dear Soers, what is your opinion on this?

Answer 1

It depends on how you model the domain of your program.

If you model domains in terms of data stored in data structures and methods that read data from one data structure and store derived data in another data structure (procedures or functions depend on your procedural or functional design), then the layout of the objects is not suitable. So-called "state" testing is what you want. The result you care about is that the procedure puts the right data in the right variables and what it calls for this to happen is just an implementation detail.

If you model domains in terms of message transfer protocols with which objects interact, then protocols are what you need and what data objects store to coordinate their behavior in the protocols in which they play roles - this is just an implementation in detail. In this case, mock objects are the right tool for specifying and checking based on state, closely linking tests to minor implementation details.

And most object-oriented programs have a combination of styles. Some code will be written exclusively functionally, transforming immutable data structures. Another code will coordinate the behavior of objects that change their hidden internal state over time.

Regarding high test coverage, that really doesn't say much. A low test coverage indicates where you have poor testing, but a high test coverage does not indicate that the code is adequately tested. Tests can, for example, go through code paths and thus increase coverage statistics, but actually do not make any statements about what these code codes did. In addition, it is important how the different parts of the program behave in combination, which the unit test will not tell you about. If you want to verify that your tests really test the behavior of your system, you can use the Mutation Testing tool. This is a slow process, so this is what you run in the nightly build, and not with every registration.

Answer 2

I read 2 questions:

What is your opinion on testing that specific methods on components are called in a specific order?

In the past, I cheated on this. These days we use a lot more “stubbing” and a lot less “ridicule”. We are trying to write unit tests that test only one thing. When we do this, you can usually write a very simple test that drowns out interactions with most other components. And we very rarely establish order. This helps make the tests less fragile.

Tests that test only one thing are easier to understand and maintain.

In addition, if you have to write a lot of expectations for interacting with a large number of components, there may well be a problem in the code that you are testing in any case. If you find it difficult to maintain tests, frequently-checked code can be reorganized.

Should be obsessed with test coverage?

When writing unit tests for this class, I'm pretty obsessed with testing. This makes it easy to spot important bits of behavior that I have not tested. I can also judge which bits I don't need.

General unit test coverage statistics? Not particularly interested as long as they are tall.

100% unit test system-wide coverage? Not interested at all.

Answer 3

I agree - I tend to lean towards checking the status, rather than checking the behavior (free interpretation of the classic TDD , while using test twins).

The Art of Unit Testing book has many good tips in these areas.

100% testing coverage, GUI testing, testing of getters / seters or other non-logic code, etc., seems unlikely to provide a good return on investment. TDD will provide high test coverage anyway. Check what might break.

Answer 4

I asked a similar question. How many module tests is a good thing , which can help give an idea of ​​the different levels of testing that people find appropriate.

Answer 5

• What is the likelihood that some junior employee will break part of the code that runs the “controller calls such and such methods in a specific order” during your code maintenance?

• What is the value of your organization if such a thing arises - in the event of a production failure, debugging / correction / retesting / re-release, legal / financial risk, reputation risk, etc. ??

Now multiply # 1 and # 2 and make sure your reluctance to reach a reasonable amount of test coverage is worth the risk.

Sometimes it will not (this is why when testing there is the concept of a point of decreasing returns).

eg. if you support a web application that is not critical to production, and it has 100 users who have a workaround if the application is broken (and / or can make a simple and immediate rollback), then spend 3 months on full coverage testing this application is -sensical.

If you are working on an application where a small mistake can have millions or worse consequences (think of a space shuttle software or cruise missile guidance system), then thorough testing with full coverage becomes much more sensitive.

Also, I'm not sure that I read too much in your question, but you seem to imply that using unit testing with corrections somehow excludes functional testing of the application / integration. If so, you are right to object to such a concept - two testing approaches must coexist.