TDD - When can I write a test without failure?

Question

From what I understand, in TDD you first need to write a failed test, and then write the code so that it passes, and then refactor. But what if your code already considers the situation you want to test?

For example, suppose I have a TDD sorting algorithm (this is just hypothetical). I can write unit tests for several cases:

input = 1, 2, 3
output = 1, 2, 3

input = 4, 1, 3, 2
output = 1, 2, 3, 4
etc...

To complete the tests, I complete using a quick dirty bubble. Then I reorganize and replace it with a more efficient merge sort algorithm. Later, I understand that we need it to be stable, so I also wrote a test. Of course, the test will never fail, because merge-sort is a stable sorting algorithm! Despite this, I still need this test if someone reorganizes it again to use another, possibly unstable sorting algorithm.

Does this violate the TDD mantra by always writing failed tests? I doubt anyone would recommend that I spend time introducing an unstable sort algorithm just to test a test case, and then override merge sort. How often do you come across a similar situation and what do you do?

Answer 1

At first, there are two reasons for writing failed tests, and then running them;

First, check to see if the test really checks what you are writing it for. First you check to see if it works, you changed the code to run a test run, then check to see if it runs. This seems silly, but I had a few cases when I added a test for the code, which I already managed to find out later, that I made a mistake in the test, which made it always run.

The second and most important reason is to prevent you from writing too many tests. Tests reflect your design, and your design reflects your requirements and requirements. You do not want to rewrite a lot of tests when this happens. A good rule of thumb is that each test fails for only one reason, and for this only one test fails. TDD tries to do this by repeating the standard red-green-refactor cycle for every test, every function, and every change in your codebase.

But, of course, the rules are being violated. If you remember why these rules are made in the first place, you can be flexible with them. For example, when you find that you have tests that test more than one thing, you can separate it. Effectively, you wrote two new tests that you have not seen before. Breaking and then fixing the code to see how your new tests fail is a good way to double-check the situation.

Answer 2

I doubt anyone would recommend me time to implement an unstable sorting algorithm to check a test case, then override Merge Sort. How often do you come across a similar situation and what do you do?

Let me recommend it then. :)

All of these are trade-offs between the time you spend, on the one hand, and the risks that you reduce or decrease, and also understand what you get, on the other hand.

Continuing a hypothetical example ...

If "stability" is an important property / function, and you do not "test the test", if it does not work, you save the time it takes to complete this work, but do not risk that the test is incorrect and will always turn green.

If, on the other hand, you “test a test” by disrupting a function and observing how it fails, you reduce the risk of the test.

And a wildcard, you can gain some important knowledge. For example, when trying to code a “bad” sort and make the test fail, you might think more deeply about the comparison restrictions for the type you are sorting and find that you used “x == y” as a predicate of the equivalence class for sorting, but in fact, "! (x <y) & &! (y <x)" is the best predicate for your system (for example, you may find a design error or defect).

So, I say that the mistake is on the side of “spending extra time so that it fails, even if it means intentionally breaking the system, to get a red dot on the screen for a moment,” because although each of these little “sabotages” carries some time cost, each time while one of them will save you a huge package (for example, oops, an error in the test means that I have never tested the most important property of my system, or, unfortunately, our entire project for the inequality of the predicates are mixed up) . It is like playing a lottery, except in the long run the odds are in your favor; every week you spend $ 5 on tickets, and you usually lose, but every three months you win a jackpot of $ 1,000.

Answer 3

The only big advantage of running the test first is that it ensures that your test really checks what you think. You may have subtle errors in your test that cause it to not test at all.

For example, I once saw in our C ++ code base someone was testing a test:

assertTrue(x = 1); 

Obviously, they did not program so that the test would fail first, since it does not test anything.

Answer 4

If you write a new piece of code, you write a test, and then code. This means that the first time you always run a test with an error (because it runs against the dummy interface), then you can reorganize several times, in which case you may not need to write additional tests, because the one that you are, maybe already enough.

However, you can save some code using TDD methods; in this case, you first need to write the tests as performance tests (which, by definition, will never run because they run with working interfaces), then refactoring.

Answer 5

A simple TDD rule: you write tests that may fail.

If a software developer told us anything, it means that you cannot predict test results. It didn’t even work. In fact, for me quite often there are “new feature requests” that already work in existing software. This is common, as many of the new features are direct extensions to existing business desires. Basic, orthogonal software design will continue to work.

those. The new function "List X must contain up to 10 elements" instead of "up to 5 elements" will require a new test case. The test passes when the actual implementation of List X allows you to use 2 ^ 32 elements, but you do not know this for sure until you run a new test.

Answer 6

Hard TDDers stated that you always need a test failure to verify that a positive test is not false, but I think that in reality many developers pass a failed test.

Answer 7

There are reasons to write tests in TDD beyond just “test” development.

Suppose your sorting method has some other properties outside of a direct sorting action, for example: it checks that all inputs are integers. You do not initially rely on this, and it is not in the specification, so there is no test.

Later, if you decide to use this extra behavior, you need to write a test so that anyone else who comes in and refactors does not violate this extra behavior that you now rely on.

Answer 8

uh ... I read the TDD loop as

• first write a test that fails because the code is just a stub
• write code to pass the test
• refactoring if necessary

there is no obligation to continue to write tests that fail; the first one fails because there is no code for anything. point of the first test is the choice of interface!

EDIT: There seems to be some misunderstanding of the red-green refactor mantra. According to wikipedia TDD article

In test development, every new feature begins with writing a test. This test inevitably fails because it was written before the function was implemented.

In other words, the failure test should be for a new function, not for additional coverage!

EDIT: if you are not talking about writing a regression test to reproduce the error, of course!

Answer 9

The example you provided is IMO one of the right times to write a test that passes first. The goal of proper tests is to document the expected behavior of the system. It’s normal to write a test without changing the implementation to further figure out what the expected behavior is.

PS

As I understand it, here is the reason why you want the test to fail before skipping it:

The reason you “write a test that you know will fail, but test it before you miss it,” is because from time to time the initial assumption that the test will surely fail will fail . In these cases, the test now saved you from writing unnecessary code.

Answer 10

As others have said, the TDD mantra is “no new code, no unit test failure”. I have never heard TDD practitioners say "no new tests without missing code." New tests are always welcome, even if they "accidentally" pass "randomly." There is no need to change the code for the break, and then change it to pass the test.

Answer 11

But what if your code already considers the situation you want to test?

Does this violate the TDD mantra by always writing failed tests?

Yes, because you have already broken the mantra of writing a test before the code. You can simply delete the code and start over, or simply accept a test that works from the beginning.

Answer 12

I have come across this situation many times. Although I recommend and try to use TDD, sometimes it interrupts the stream too much to stop and write tests.

I have a two step solution:

• After you have the working code and your test without failures, deliberately insert changes into the code to cause the test to fail.
• Cut this change from the source code and put it in a comment - either in the code method or in the test method, so the next time someone wants to make sure that the test is still gaining failures, they know what to do. It also serves as proof that you have confirmed that the test is failing. If it is clean, leave it in the code method. You might even want to do this to use conditional compilation to turn on test switches.