Is Assert.Fail () considered bad practice?

Question

I use Assert.Fail a lot when doing TDD. I usually work on one test at a time, but when I get ideas about things that I want to implement later, I quickly write an empty test where the name of the testing method indicates what I want to implement as a kind of to-do list. To make sure I don't forget, I put Assert.Fail () in the body.

When checking xUnit.Net, I found that they did not implement Assert.Fail. Of course, you can always Assert.IsTrue (false), but this also does not communicate my intention. I got the impression that Assert.Fail was not implemented specifically. Is this considered bad practice? If so, why?

---

@ Martin Meredith This is not exactly what I am doing. First I write a test, and then I implement the code to make it work. I usually think of several tests at once. Or I think of testing when I work on something else. This is when I write an empty failure test to remember. By the time I start writing the test, I am carefully working on testing.

@Jimmeh This seems like a good idea. Ignored tests fail, but they still appear in a separate list. Have to try it.

@Matt Howells Great idea. NotImplementedException binds intent better than assert.Fail () in this case

@ Mitch Wheat This is what I was looking for. It seems that it was forbidden so as not to abuse in another way, I am abusing it.

Answer 1

For this scenario, and not for calling Assert.Fail, I do the following (in C # / NUnit)

[Test] public void MyClassDoesSomething() { throw new NotImplementedException(); } 

It is more explicit than Assert.Fail.

There seems to be general agreement that it is preferable to use more explicit statements than Assert.Fail (). Most frameworks should include it, although they do not offer a better alternative. For example, NUnit (and others) provide an ExpectedExceptionAttribute to verify that some code throws a specific exception class. However, to verify that the property in the exception is set to a specific value, it cannot be used. Instead, you need to resort to Assert.Fail:

 [Test] public void ThrowsExceptionCorrectly() { const string BAD_INPUT = "bad input"; try { new MyClass().DoSomething(BAD_INPUT); Assert.Fail("No exception was thrown"); } catch (MyCustomException ex) { Assert.AreEqual(BAD_INPUT, ex.InputString); } } 

The xUnit.Net Assert.Throws method makes this much neater without requiring the Assert.Fail method. Not including the Assert.Fail () method, xUnit.Net encourages developers to find and use more explicit alternatives and to support the creation of new claims where necessary.

Answer 2

This was intentionally omitted. This is Brad Wilson's answer about why there is no Assert.Fail () argument:

In fact, we did not notice this. I find Assert.Fail - this is a crutch that implies that there is probably a missing statement. Sometimes it's just a way to structure the test, and sometimes it's because Assert use a different statement.

Answer 3

I always used Assert.Fail () to handle cases when you found that the test should fail with logic, except for a simple comparison of the values. As an example:

 try { // Some code that should throw ExceptionX Assert.Fail("ExceptionX should be thrown") } catch ( ExceptionX ex ) { // test passed } 

Thus, the absence of Assert.Fail () in the structure seems to me an error. I would suggest fixing the Assert class to include the Fail () method, and then sending the patch to the developers of the framework along with your arguments for adding it.

As for your practice of creating tests that deliberately fail in your workspace to remind yourself how to implement them before committing, this seems like a great practice to me.

Answer 4

I use MbUnit for my unit testing. They have the ability to ignore tests that appear in orange (rather than green or red) in the test suite. Perhaps xUnit has something similar, and will mean that you don’t even have to enter any statements in this method, because it will appear in an annoyingly different color, which makes it hard to miss?

Edit:

In MbUnit, this happens as follows:

 [Test] [Ignore] public void YourTest() { } 

Answer 5

This is the pattern that I use when writing a test for the code that I want to create with an exception:

 [TestMethod] public void TestForException() { Exception _Exception = null; try { //Code that I expect to throw the exception. MyClass _MyClass = null; _MyClass.SomeMethod(); //Code that I expect to throw the exception. } catch(Exception _ThrownException) { _Exception = _ThrownException } finally { Assert.IsNotNull(_Exception); //Replace NullReferenceException with expected exception. Assert.IsInstanceOfType(_Exception, typeof(NullReferenceException)); } } 

IMHO is the best way to test for exceptions using Assert.Fail (). The reason for this is that I not only check that an exception is thrown, but also check the type of exception. I understand that this is similar to the answer from Matt Howells, but IMHO using the finally block is more reliable.

Obviously, you can still include other Assert methods to check the input line of exceptions, etc. I would be grateful for your comments and opinions on my template.

Answer 6

Personally, I have no problem using a test suite as a todo list like this, until you eventually bypass writing a test before embedding the code you pass.

Having said that, I used this approach myself, although now I find that it leads me to write too many tests, which strangely seems like the inverse problem of not writing tests at all: you end up making design decisions too soon IMHO.

By the way, in MSTest, the standard test pattern uses Assert.Inconclusive at the end of its samples.

AFAIK. The xUnit.NET framework should be extremely lightweight and yes, they did truncate Fail intentionally to encourage the developer to use an explicit failure condition.

Answer 7

Wild hunch: holding Assert.Fail is meant to prevent you from thinking that a good way to write test code is a huge pile of spaghetti leading to Assert.Fail in bad cases. [Edit to add: other people's responses generally confirm this, but with quotes]

Since this is not what you are doing, it is possible that xUnit.Net is overly protective.

Or maybe they just find it so rare and so unorthogonal that they are not needed.

I prefer to implement a function called ThisCodeHasNotBeenWrittenYet (actually something shorter, for ease of input). It is not possible to communicate the intention more clearly than this, and you have an exact search query.

If this fails or is not implemented (provoke a linker error) or a macro that does not compile, you can change it to suit your current preferences. For example, if you want to run something that was finished, you need a crash. When you sit down to get rid of them, you may get a compilation error.

Answer 8

With good code, I usually do:

 void goodCode() { // TODO void goodCode() throw new NotSupportedOperationException("void goodCode()"); } 

Using test code, I usually do:

 @Test void testSomething() { // TODO void test Something Assert.assert("Some descriptive text about what to test") } 

If you are using JUnit and don't want to get a crash, but an error, then I usually do:

 @Test void testSomething() { // TODO void test Something throw new NotSupportedOperationException("Some descriptive text about what to test") } 

Answer 9

Beware of Assert.Fail and its corrupting effects so that developers write dumb or broken tests. For example:

 [TestMethod] public void TestWork() { try { Work(); } catch { Assert.Fail(); } } 

This is stupid because the try-catch attempt is redundant. A test fails if it throws an exception.

Besides

 [TestMethod] public void TestDivide() { try { Divide(5,0); Assert.Fail(); } catch { } } 

This is broken, the test will always pass regardless of the result of the Divide function. Again, the test fails if and only if it throws an exception.

Answer 10

Why are you using Assert.Fail to say that there should be an Assert.Fail exception? That is unnecessary. Why not just use the ExpectedException attribute?

Answer 11

If you write a test that just fails and then writes code for it, then write a test. This is not a test development.

Technically, Assert.fail () is not required if you are using test development correctly.

Do you consider using Todo List or applying the GTD methodology to your work?

Answer 12

MS Test has Assert.Fail () , but also has Assert.Inconclusive () . I think the most appropriate use for Assert.Fail () is if you have some built-in logic that would be inconvenient for approval, although I can't even come up with any good examples. For the most part, if your test environment supports something other than Assert.Fail (), then use this.

Answer 13

I think you should ask yourself what (premature) testing should do.

First you write a (set) test without implantation. Perhaps also rainy day scenarios.

All these tests must be unsuccessful in order to be correct tests: So, you want to achieve two things: 1) Make sure that your implementation is correct; 2) Check that your device checks correctly.

Now, if you are doing upfront TDD, you want to run all your tests, as well as parts of NYI. The result of your overall test run passes if: 1) All materials sold succeed 2) All NYI information does not work

In the end, it would be a unit test ommision if your device tests were successful until there is an implementation, right?

You want to receive something like the mail of your continuous integration test, which checks all implemented and non-implemented code and is sent if any implemented code fails or any non-implemented code succeeds. Both are undesirable results.

Just write tests [ignore] that will not do the job. None of the statements that stops the first failure does not run other tests in the test.

Now how to do it? I think this requires a more advanced organization of your testing. And this requires some other mechanism to achieve these goals.

I think you need to separate your tests and create some tests that will run completely, but should fail, and vice versa.

The ideas are to split your tests into several assemblies, use a grouping of tests (ordered tests in mstest can do the job).

However, CI builds that letters, if not all tests in the NYI department, are not easy and straightforward.