Should a JUnit message indicate success or failure?

I can write an approval message in one of two ways. Ad Success:

assertEquals( "objects should be identical", expected, actual ); 

Or stating a condition of violation:

 assertEquals( "objects aren't identical", expected, actual ); 

Is there a standard for this in JUnit? If not, what are the arguments for each side?

PS I saw articles on the Internet showing them without explanation, so just saying "search Google" is not the answer!

[UPDATE]

Everyone is obsessed with the fact that I used assertEquals , and therefore the message is probably useless. But, of course, only because I just wanted to illustrate the issue.

So imagine this:

 assertTrue( ... big long multi-line expression ... ); 

If the message is helpful.

+51
java unit-testing junit
Jul 02 '09 at 15:05
source share
12 answers

I rarely even worry about the message, at least for assertEquals . Any reasonable tester will explain that you used assertEquals and two things that should have been equal. None of your posts provide more information.

I usually find that unit test failures are transient things - I quickly find out what's wrong and fix. "Finding out what's wrong" usually includes enough details that one message will not make much difference. Consider the "time saved by the presence of a message" and the "time spent thinking messages" :)

EDIT: Well, one case where I could use a message: when there is a compact description in the text that is not obvious from the string representation of the object.

For example: "Expected date should be December 1" when comparing dates stored in milliseconds.

I would not worry about how you express it, though: just make sure this is obvious from the message you mean. Either "it should be" or "it wasn’t" —that's good — just December 1 will not be obvious.

+29
Jul 02 '09 at 15:08
source share

According to the junit API, the message is an "identification message for AssertionError", so this is not a message describing the condition that must be met, but a message that describes what is wrong if the condition is not met. Therefore, in your example, “objects are not identical,” appear to be more compatible.

+17
Dec 15 '12 at 18:08
source share

Unlike many others, I feel that using a message is extremely useful for many reasons:

  • A person viewing the logs of a failed test may not be the person who wrote the test. It may take some time to read the code and understand in which case this statement is intended to solve. A useful message will save time.

  • Even if the test developer is looking through the logs, it may be days or months from the moment of writing the test, and again the message can save time.

My advice would be to write a message expressing expected behavior. For example:

 assertEquals("The method should be invoked 3 times", 3, invocationCount); 
+6
Jul 28 '12 at 5:24
source share

I do not think it matters at all. You already know that a failure has occurred, and therefore it doesn’t matter if the message says what was supposed to happen or what should not.

The purpose of the message is to help you when possible, and not to gain some completeness.

Obviously, in the case of assertEquals this is less important, but the message is important in the case of general statements. The message should help you get enough context to immediately understand what exactly failed.

However, the amount of context needed (and therefore the details in the message) should depend on how you get the report. For example, if you get it in Eclipse, you can easily go and interact and see what happened, so the message is less important. However, if you send your reports by e-mail (for example, from a continuous build server), you want the message to provide enough information so that you have an idea of ​​what is happening before you go to the corresponding source code.

+3
Jul 02 '09 at 15:19
source share

I would like to answer the question without considering whether the message is useful in generel.

If the test fails, something is wrong. I know it. I want to know why it is broken. It is very easy to find out, because I just have to open a test case and SUT. As John said, it's very easy to fix this (hopefully ;-)).

But what about the message? The message for me is advice on what you can do to turn it into a green test case. Therefore, I would be grateful if the message text included recommendations on how to fix this problem or where to look for the problem.

Another interesting aspect is the use of positive expressions. It is worth considering the use of positive text messages. In your example, I would use Objects should be identical . But this is a small reason.

+2
Jul 03 '09 at 5:45
source share

Vote also (e.g. Jon), but the only time I've ever used such a message (on assert equals) is to create one test with a matrix of values ​​and one of the test elements: I use the message to indicate which test case ended unsuccessfully. Otherwise, the text is completely redundant.

+1
Jul 02 '09 at 15:22
source share

From javadocs JUnit:

Claims that two objects are equal. If they are not an AssertionFailedError is thrown with this message.

According to the API, the message can be anything you want. I would say that the two options that you have are the same and both are superfluous. The success or failure of assert already provides all the information that you provide in the message.

It follows that you should either have nothing (there is a statement that does not accept a whole line) OR include a message with a value that goes beyond what already exists.

So, I think this is a repetition of John's answer, but too detailed to be a comment.

+1
Jul 02 '09 at 15:23
source share

I see this question from two sides,

First, the most common perspective, which most of us are already talking about: From the point of view of the one who sees the logs and tries to correct the error: I believe that both messages provide equal information.

The second perspective is the one who reads / maintains / looks at the code: As we talked with age about the readability and simplicity of the code. Thus, it is also no less important.

We are convinced that my code should be simple and self-evident, so that there are no explicit comments, and I strongly agree with it.

From this point of view:

These messages make reading and viewing code easier, as they serve the dual purpose of documentation, as well as reporting errors:

 assertEquals( "objects should be identical", expected, actual ); assertTrue( "flag should have been set", flag ); assertNotNull( "object must not be null", object ); 

These posts are not so reader friendly as they speak of an unexpected state:

 assertEquals( "objects aren't identical", expected, actual ); assertTrue( "flag is not set", flag ); assertNotNull( "object is null", object ); 
+1
Dec 20 '15 at 5:59
source share

I do not put a message for the case that you are quoting if I do not run the test, where I have an array of similar test values ​​that I run in the loop, and I want to determine exactly which one was unsuccessful. Then I add a message to tell me which one.

0
Jul 02 '09 at 21:04
source share

I agree that providing a message is helpful, and I always provide it.

A useful thing for me is a clear statement about what went wrong - usually with the words “should” or “should not”.

For example, “objects are equal” are ambiguous - does this mean that the objects are equal and why the test failed? Or that the objects should be equal, but is it not? But if you say: “Objects must be equal” or “Objects must not be equal”, this is obvious why the statement did not work.

0
Mar 20 '14 at 21:56
source share

I especially like the way Spock's test environment encourages tests that read like history, and similarly come up with structural testing in different contexts. I am not particularly worried about the individual error message, which makes a lot of sense, I tried to quickly wrap my head around the entire test as soon as I open it:

 assertEquals("Cloned and persisted items", mockedTiCount, clonedTis.size()); assertTrue("Belong to new transaction", clonedTis.stream().allMatch(ti -> ti.getTransaction().getId().equals(cloned.getId()))); assertNotEquals("Which has a different ID", t.getId(), cloned.getId()); assertEquals("While the originals are left intact", mockedTiCount, transactionItemRepository.findByTransactionId(t.getId()).size()); 

Optimization of many small tests instead of several large ones helps here, as well as a neatly structured, hopefully reusable test installation code.

0
Jul 23 '16 at 9:05
source share

According to the specifications, the message should describe the error when this occurs. And it is useful when you create an application in the CI environment, for example, Jenkins, and use plugins to analyze the error results.

http://junit.sourceforge.net/javadoc/org/junit/Assert.html#assertTrue (java.lang.String, boolean)

 Parameters: message - the identifying message for the AssertionError (null okay) condition - condition to be checked 
0
Nov 30 '16 at 13:40
source share



All Articles