Osherove convention for negative unit tests?

Question

I am trying to make a naming decision for unit tests. I like what Roy Osherov recommended:

[MethodName_StateUnderTest_ExpectedBehavior]

http://osherove.com/blog/2005/4/3/naming-standards-for-unit-tests.html

I am not sure about this standard for negative tests, where we test whether it handles incorrect behavior correctly by throwing an exception.

So in this case, the ExpectedBehavior will always be "CorrectExceptionThrown". Would it make more sense to write ExpectedBehavior for each negative unit test, or is it okay to make it optional?

There are pros and cons. On the one hand, this is always the same for negative tests, so it would be superfluous to write it every time, this makes the unit test method name long. It is also dangerous that the expected behavior will not be added for unit test, where necessary, if we make this optional. I also think that it is better to keep it consistent throughout the project, so apply it the same way as everywhere else.

Answer 1

There is nothing superfluous in determining which exception is selected as the result of an operation. It really fits perfectly with Roy's naming convention, given:

SomeMethod_ExpectionalState_ThrowsInvalidOperationException SomeMethod_ExceptionalState_ThrowsArgumentNullException 

You get important information about your type of excluded code type. However, when you have a classic happy path test, the usefulness of some parts of the name is subjective. Consider:

 SomeMethod_DependencyReturnsCorrectResult_ReturnsResult SomeMethod_WhenNothingSpecialHappens_ReturnsResult SomeMethod_EverythingElseWorked_WorksToo 

What information do such names carry? Pretty little. ReturnsResult essentially means that it works. NothingSpecialHappens also pretty vague information. In such cases, dropping part of the name may be justified.

Please note, however, instead, it might be worth considering changing the name rather than completely discarding part of it (for example, ReturnsResult may be replaced by a less vague ReturnsEntityFromDatabase or ReturnsSerializedValue ).

Finally, do not follow Roy blindly - treat him as recommendations , not conventions. Conventions are rarely suitable for all possible situations, and this is no different.

Answer 2

You can write:

 [Test] public void Foo_ExceptionalCaseX1_ExceptionY1Thrown() { } [Test] public void Foo_ExceptionalCaseX2_ExceptionY2Thrown() { } ... 

If the Exceptions are different, and the type of exceptions is the same, then there is no redundancy (even if the suffix is ​​the same). This is no different from writing these two tests:

 [Test] public void Foo_SomeCaseX1_42Returned() { } [Test] public void Foo_SomeCaseX2_42Returned() { } ... 

What you can do - 42 returns in two cases, and this reality is the same for exceptions.

And one more thing: when you read the list of tests, they may look (almost) the same, but when one of them fails one day, a successful developer will be able to immediately find out what was the expected behavior. Each test should stand for itself.

Answer 3

Imagine a situation where one unit test fails. You will then receive a message from CI:

 Failed unit tests: MethodName_NegativeTestParams1_CorrectExceptionThrown 

And no other context. And you see the problem (wrong exception selected). If you make this optional or try to shorten the method name, you can end up

 Failed unit tests: MethodName_NegativeTestParams1 

without any clue what went wrong until you studied the test.

In this case, when planning a situation where you have no context other than a list of failed unit tests, you should make the method names as detailed as possible by repeating CorrectExceptionThrown as many times as necessary.

Also, a CorrectExceptionThrown message may be less general: ArgumentExceptionThrown , etc., if you have different exceptions for different tests.

Thus, I would include the expected behavior in all cases, although sometimes this may seem like an unnecessary repetition.

Answer 4

I think it is true; however, this is just one way to resolve the unit test naming convention. If you obviously know the exception throw in this test, I think testing the modular .NET platform will provide an easy way, as in JUnit.

@Test (IOException.class) public void testIOException() {...}