Exclude individual JUnit Test methods without changing the Test class?

Question

I am currently reusing JUnit 4 tests from another project against my code. I get them directly from another project repository as part of my automatic Ant build. This is great, as it ensures that I keep my code green against the latest version of the tests.

However, there are a subset of tests that I never expect to pass to my code. But if I start adding @Ignore annotations to these tests, I will have to keep my own separate copy of the test implementation, which I really don't want to do.

Is there a way to exclude individual tests without changing the source of testing? Here is what I watched so far:

• As far as I can tell, the Ant JUnit task allows you to exclude entire test classes, rather than individual test methods - so I'm not good, I need the granularity of the method.

• I thought about building TestSuite, which uses reflection to dynamically find and add all the source tests, and then add code to explicitly remove those tests that I don't want to run. But I discarded this idea when I noticed that the TestSuite API does not provide a method for removing tests.

• I can create my own test classes that extend the original test classes, override those tests that I don't want to run, and annotate them with @Ignore. Then I run JUnit in my subclasses. The disadvantage here is that if new test classes are added to the original project, I will not automatically select them. I will have to keep track of new test classes as they are added to the original project. This is my best option, but it is not perfect.

• The only other option I can think of is to run failed tests anyway and ignore the failures. However, these tests take time to run (and do not work!), So I would rather not run them at all. In addition, I see no way to tell Ant tasks to ignore errors with certain testing methods (again - I see how you can do this for individual test classes, but not methods).

Answer 1

If you can not touch the original test, you will have serious limitations. Your main sound sounds like the best bet, but with a few changes:

Create Ant tests, excluding superclasses, so that extra classes that you don't know about are run.

You can use the @Rule annotation (new to JUnit 4.7) to know which test is running and abort it (returning an empty Statement implementation), instead of overriding specific methods, which gives you more flexibility in understanding whether to avoid the test. The only problem with this method is that you cannot stop the @Before methods from using this method, which can be slow. If this is a problem (and you really can't touch the tests), then @Ignore in the overridden method is the only thing I can think of.

If, however, you can touch on these tests, some additional options will open:

You can run them using a special runner by specifying the @RunWith tag in the class. This runner will simply execute the standard runner (JUnit4.class) in this project, but in your project (through a system property or some other mechanism) it will check the name of the test and not run the test. This has the advantage of being the least intrusive, but the most difficult to implement (runners are hairy animals, one of @Rule's stated goals is to eliminate most of the need to create them).

Another is to create an acceptThat statement in a test that will check some configuration parameters that would be true if this test should be run. This actually involves injecting directly into the test, which is most likely a transaction violator in everything that is remotely referred to as a “separate project”.

Answer 2

Now this will not help, but TestNG supports such features.

Answer 3

Well, this is a pretty tough decision, but don't throw things at me if that sounds funny.

The core of Junit4 is the org.junit.runner.Runner class and its various subclasses, most importantly org.junit.runners.Suite . These runners determine which tests are designed for a given test class, using things like @Test and @Ignore.

It is very easy to create custom runner implementations, and they can usually be linked using the @RunWith annotation on your test classes, but obviously this is not an option for you.

However, in theory, you could write your own Ant task, perhaps based on the standard Ant Junit task, which takes your own test runner and uses it directly, passing each test class to it in turn. Your runner's implementation may use an external configuration file that indicates which test methods to ignore.

It would be a lot of work, and you would have to spend time looking in the prehistoric Ant code base Junit to find out how it works. However, investing on time can be worth it.

It's just a shame that the Junit Ant task does not provide a mechanism for specifying a test runner that would be ideal.

Answer 4

An opportunity that I can come up with to achieve what you want, with the indicated limitations, is to use a modification of the byte code. You can save the list of classes and methods to ignore in a separate file and correct the bytecode of the test classes when they are loaded to completely remove these methods.

If I'm not mistaken, JUnit uses reflection to find testing methods to run. The method rename operation will allow you to remove these methods before JUnit finds them. Or, the method can be changed to return immediately without performing any operations.

A library like BCEL can be used to change classes at boot time.

Answer 5

If you want to run only a subset of the tests, it sounds as if this class has more than one responsibility and needs to be reorganized. Alternatively, you can break the test class so that the original project has all the tests, but in one or more classes (I assume that some of the tests are really integration tests and relate to the database or network), and you can exclude the class (es) I wanted to.

If you cannot do this, your override option is probably best. Take the process when you need to ignore some methods that you extend for this class, and add it to the Ant exception list. This way you can eliminate the fact that you cannot pass, and will continue to use all new tests (methods that you have not redefined, and new test classes) without changing your assembly.

Answer 6

If unwanted tests relate to specific classes / packages, you can use file exclusion in Ant to exclude them during import.

Answer 7

Two options

• Work with the owner of the borrowed tests to extract them into a separate class that you both can share.
• Create your own test class that proxies the test class that you want to use. For each method that you want to include, there is a method in your class. You will need to create an instance of the test class that you call, and do the before and after methods too, if they are in the original.
• Create a custom Junit runner based on blockjunitrunner and use it to filter or in the themes you want.