Maven + Surefire return code is 0 for failed tests

Question

I have a project with tests divided into units and phases of integration. I have a working buildbot, and the problem is that even in the tests the maven return code crashes equal to 0, so the buildbot build is successful.

This is the result of the mvn integration test:

Results : Tests in error: Info about failed tests Tests run: 5, Failures: 0, Errors: 5, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1 minute 19 seconds [INFO] Finished at: Tue Feb 12 09:43:53 UTC 2013 [INFO] Final Memory: 36M/97M [INFO] ------------------------------------------------------------------------ $ echo $? 0 

The result for installing mvn is the same without a successful build assembly. Results:

 Tests in error: Info about failed tests Tests run: 5, Failures: 0, Errors: 5, Skipped: 0 $ echo $? 0 

Surefire configuration is as follows:

 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.13</version> <configuration> <printSummary>true</printSummary> <excludedGroups>com.testlib.IntegrationTest</excludedGroups> </configuration> <executions> <execution> <id>unit-tests</id> <phase>test</phase> <goals> <goal>test</goal> </goals> <configuration> <excludedGroups>com.testlib.IntegrationTest</excludedGroups> </configuration> </execution> <execution> <id>integration-tests</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <includes> <groups>com.testlib.IntegrationTest</groups> </includes> </configuration> </execution> </executions> </plugin> 

I read other topics about maven return codes, but in theory, related errors should be fixed in my version of maven (Apache Maven 2.2.1 (rdebian-8))

Is there any way to change this behavior?

Update: As I said, I tried:

 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.13</version> <dependencies> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-junit47</artifactId> <version>2.13</version> </dependency> </dependencies> <configuration> <groups>com.testlib.IntegrationTest</groups> </configuration> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> <configuration> <includes> <include>**/*.class</include> </includes> </configuration> </execution> </executions> 

I need the surefire-junit command to avoid initialization errors.

Answer 1

I managed to get it to work with two different configurations using groups and using folders using only surefire

 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.13</version> <configuration><excludedGroups>com.testlib.IntegrationTest</excludedGroups> </configuration> <executions> <execution> <id>integration-test</id> <phase>integration-test</phase> <goals> goal>test</goal> </goals> <configuration> <skip>false</skip> <excludedGroups>com.biicode.testlib.UnitTest</excludedGroups> <groups>com.testlib.IntegrationTest</groups> </configuration> </execution> </executions> 

  <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <excludes> <exclude>**/integration/*.java</exclude> </excludes> </configuration> <executions> <execution> <id>integration-test</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <skip>false</skip> <excludes> <exclude>none</exclude> </excludes> <includes> <include>**/integration/*.java</include> </includes> </configuration> </execution> </executions> </plugin> 

Answer 2

First check if there is the same parent pom that config:

 <testFailureIgnore>true</testFailureIgnore> 

anywhere ... you can check this via:

 mvn help:effective-pom 

In addition, you are trying to run an integration test with the maven-surefire-plugin , which is simply wrong. For integration tests, use the maven-failsafe-plugin . Another thing is to name your integration tests in the right way, for example, IT * .java, * IT.java, etc.

Another reason why you are using such an old version of Maven is version Maven 3.0.4.

Ah, sorry. Watch what you say about integration tests. If you correctly use the maven-failafe-plugin for integration tests, it contains a specific verify goal which is intended to verify the results of integration tests later. But you need to separately configure this using the execution unit and binding to a specific phase of the life cycle.