Jenkins Maven Builds Success Reports Despite Unit Test Failure

Question

Im using Maven 3.2.3, JUnit 4.12 and SureFire 2.18.1 in Jenkins. I have a multi-module Maven job configured with the following Maven options:

clean install -U -e -P qa -Dci -DskipITs 

However, when I run my Maven build, although one of my JUnit tests fails, the Maven module itself reports "SUCCESS" (output after) ...

 Executing Maven: -B -f /var/lib/jenkins/workspace/subco/subco/pom.xml clean install -U -e -P qa -Dci -DskipITs … ——————————————————————————— TESTS ------------------------------------------------------- Running org.mainco.subco.user.domain.UserTest Tests run: 24, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec <<< FAILURE! - in org.mainco.subco.user.domain.UserTest testHasRoleNullRoles(org.mainco.subco.user.domain.UserTest) Time elapsed: 0.006 sec <<< FAILURE! java.lang.AssertionError: Failed to indicate we found right role for user. at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at … [INFO] Tests are skipped. [JENKINS] Recording test results [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ othersubmodule --- [INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war [INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] subco ....................................... SUCCESS [4.849s] [INFO] database .......................................... SUCCESS [18.340s] [INFO] session ........................................... SUCCESS [6.601s] [INFO] main .............................................. SUCCESS [8:00.887s] [INFO] mainclient ........................................ SUCCESS [4:01.490s] [INFO] orders ............................................ SUCCESS [50.831s] [INFO] SBD myproject Webapp .................................. SUCCESS [1:45.484s] [INFO] subco Admin Module .......................... SUCCESS [1:01.318s] [INFO] Other Submodule ................................ SUCCESS [6.162s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 16:20.351s [INFO] Finished at: Thu Aug 20 21:17:57 UTC 2015 [INFO] Final Memory: 76M/748M [INFO] ------------------------------------------------------------------------ Waiting for Jenkins to finish collecting data [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/pom.xml to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/target/orgsclient.war to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.war [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/pom.xml to org.mainco.subco/subco/83.0.0-SNAPSHOT/subco-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/pom.xml to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/target/myproject.war to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.war [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/pom.xml to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/target/database-83.0.0-SNAPSHOT.jar to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.jar [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/pom.xml to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/target/sbadmin.war to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.war [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/pom.xml to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.jar [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main-tests.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT-tests.jar [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/pom.xml to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/target/orders.jar to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.jar [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/pom.xml to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.pom [JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/target/session.jar to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.jar Sending e-mails to: [e-mail] [e-mail] [e-mail] [e-mail] [e-mail] 

the channel is stopped Sending emails to the address: [e-mail] Warning: you do not have plugins that provide access control for assemblies, so returning to obsolete behavior allows you to start any subsequent assemblies Finished: UNSTABLE

How do I get a Maven module to report an error if one of the module tests fails? I understand general creation reports as UNSTABLE in Jenkins, but for readability, I would prefer SUCCESS to be displayed only if everything went well.

Edit: Here is the surefire plugin configuration

  <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.18.1</version> <configuration> <reuseForks>true</reuseForks> <argLine>-Xmx2048m -XX:MaxPermSize=512M -noverify -XX:-UseSplitVerifier ${argLine}</argLine> <skipTests>${skipAllTests}</skipTests> </configuration> </plugin> 

Answer 1

This answer is basically what @ spoonybard896 suggested in the comment.

Add -Dmaven.test.failure.ignore=false to MAVEN_OPTS . This can be added to the Build → Advanced section of your Jenkins job.

Usually this should be the default value, but with the tasks of Jenkins and Maven, this is not so . For them, this property is actually set to true unless you set it yourself.

(disclaimer: looked at the exact steps of Jenkins from this answer ).