How to configure SBT assembly to return zero exit code when test fails for Jenkins?

Question

When I run Specs2 tests in Jenkins via SBT, the assembly is flagged as failing as soon as one test fails. Since Jenkins usually distinguishes between failure to build and test failures, I want to change that.

I know that a build failure in Jenkins is detected by the SBT call exit code, which appears to return 1 as soon as at least one test fails.

What parameters do I suppose, I want to avoid changing my build.sbt(or the project as a whole) to correct this inconvenience?

Somehow I think that you can put a standard sbt project into a standard Jenkins installation and work as you like.

Answer 1

tl; dr Use testResultLoggerwith a custom logger of test results, TestsFailedExceptionwhich, in turn, sets the non- exit code 0.

Just noticed that I missed this requirement: “to avoid a change build.sbt. You can use any other file *.sbt, say, exitcodezero.sbtor ~/.sbt/0.13/default.sbtwith a custom one testResultLogger.

It turns out that starting with version 0.13.5, there is a way to have this behavior - see the Added parameter 'testResultLogger', which allows you to configure the test report in which it was born testResultLogger.

> help testResultLogger
Logs results after a test task completes.

TestResultLogger.SilentWhenNoTests, testResultLogger:

results.overall match {
  case TestResult.Error | TestResult.Failed => throw new TestsFailedException
  case TestResult.Passed                    =>
}

, TestsFailedException TestsFailedException , :

[error] Failed: Total 3, Failed 1, Errors 0, Passed 2
[error] Failed tests:
[error]         HelloWorldSpec
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful

, . build.sbt 0:

testResultLogger in (Test, test) := new TestResultLogger {
    import sbt.Tests._
    def run(log: Logger, results: Output, taskName: String): Unit = {
        println("Exit code always 0...as you wish")
        // uncomment to have the default behaviour back
        // TestResultLogger.SilentWhenNoTests.run(log, results, taskName)
    }
}

TestResultLogger.SilentWhenNoTests.run TestResultLogger.SilentWhenNoTests.run .

➜  failing-tests-dont-break-build  xsbt test; echo $?
JAVA_HOME=/Library/Java/JavaVirtualMachines/java8/Contents/Home
SBT_OPTS= -Xms512M -Xmx1536M -Xss1M -XX:+CMSClassUnloadingEnabled -Dfile.encoding=UTF-8
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Set current project to failing-tests-dont-break-build (in build file:/Users/jacek/sandbox/failing-tests-dont-break-build/)
[info] HelloWorldSpec
[info]
[info] The 'Hello world' string should
[info] x contain 11 characters
[error]    'Hello world' doesn't have size 12 but size 11 (HelloWorldSpec.scala:7)
[info]
[info] + start with 'Hello'
[info] + end with 'world'
[info]
[info] Total for specification HelloWorldSpec
[info] Finished in 15 ms
[info] 3 examples, 1 failure, 0 error
Exit code always 0...as you wish
[success] Total time: 1 s, completed Sep 19, 2014 9:58:09 PM
0

Answer 2

, script, 0. ( , , , t )

Answer 3

Based on the decision of Jacek Laskowski you can do (at least in sbt> = 1.2.8):

    testResultLogger in (Test, test) := TestResultLogger {
      (log, results, taskName) =>
        try {
          (testResultLogger in (Test, test)).value.run(log, results, taskName)
        } catch {
          case _: TestsFailedException =>
            println("Ignore TestsFailedException to get exit code 0")
        }
    }

If you have a multi-module project, you can implement it as a plugin:

object TestExitCodePlugin extends AutoPlugin {
  override def requires = JvmPlugin

  override def trigger = allRequirements

  override def projectSettings: Seq[Def.Setting[_]] = Seq(
    testResultLogger in(Test, test) := TestResultLogger {
      (log, results, taskName) =>
        try {
          (testResultLogger in(Test, test)).value.run(log, results, taskName)
        } catch {
          case _: TestsFailedException =>
            println("Ignore TestsFailedException to get exit code 0")
        }
    }
  )
}