Running JavaScript block tests without problems in continuous integration builds

Question

I have a webapp build plan that runs on a continuous integration system ( Atlassian Bamboo ). I need to include QUnit JavaScript modulation tests in the build plan so that Javascript tests are run on each assembly and Bamboo will interpret the test results.

It is advisable that I can make the build process "standalone" so that connection to external servers is not required. Good ideas on how to do this? The CI system performing the build process is located on the Linux Ubuntu server.

Answer 1

As I managed to solve the problem myself, I thought it would be nice to share it. This approach may be flawless, but it is the first that seemed to work. Feel free to post improvements and suggestions.

What I did in a nutshell:

• Launch Xvfb Instance, Virtual Framebuffer
• Using JsTestDriver :
  • run a Firefox instance in a virtual framebuffer (without a head)
  • grab a Firefox instance and run the test suite
  • generate JUnit-compatible .XML test results
• Use Bamboo to check the result file to transfer or fail assembly

Next I will talk about more detailed steps. This is what my directory structure looked like:

 lib /
     JsTestDriver.jar
 test /
     qunit /
             equiv.js
             QUnitAdapter.js
     jsTestDriver.conf
     run_js_tests.sh
     tests.js
 test-reports /
 build.xml

On the build server:

• Install Xvfb ( apt-get install Xvfb )
• Install Firefox ( apt-get install firefox )

In your application to create:

• Install JsTestDriver: http://code.google.com/p/js-test-driver/
  • add QUnit equiv.js and QUnitAdapter.js
  • configure JsTestDriver ( jsTestDriver.conf ):

 server: http: // localhost: 4224

 load:
 # Load QUnit adapters (may be omitted if QUnit is not used)
   - qunit / equiv.js
   - qunit / QUnitAdapter.js   

 # Tests themselves (you'll want to add more files)
   - tests.js

Create a script file to run unit tests and generate test results (example in Bash, run_js_tests.sh ):

 #!/bin/bash # directory to write output XML (if this doesn't exist, the results will not be generated!) OUTPUT_DIR="../test-reports" mkdir $OUTPUT_DIR XVFB=`which Xvfb` if [ "$?" -eq 1 ]; then echo "Xvfb not found." exit 1 fi FIREFOX=`which firefox` if [ "$?" -eq 1 ]; then echo "Firefox not found." exit 1 fi $XVFB :99 -ac & # launch virtual framebuffer into the background PID_XVFB="$!" # take the process ID export DISPLAY=:99 # set display to use that of the xvfb # run the tests java -jar ../lib/JsTestDriver.jar --config jsTestDriver.conf --port 4224 --browser $FIREFOX --tests all --testOutput $OUTPUT_DIR kill $PID_XVFB # shut down xvfb (firefox will shut down cleanly by JsTestDriver) echo "Done." 

Create an Ant target that calls the script:

 <target name="test"> <exec executable="cmd" osfamily="windows"> <!-- This might contain something different in a Windows environment --> </exec> <exec executable="/bin/bash" dir="test" osfamily="unix"> <arg value="run_js_tests.sh" /> </exec> </target> 

Finally, tell the Bamboo build plan both to invoke the test target and to search for JUnit test results. Here, the default is "**/test-reports/*.xml" .

Answer 2

For anyone interested in running their Jasmine BDD specifications headless in maven, you might be interested in the jasmine-maven plugin, which I support:

http://github.com/searls/jasmine-maven-plugin

Answer 3

Alternatively, you can also try TestSwarm. I ran it and ran using QUnit to run JS tests.

• http://github.com/jquery/testswarm
• http://swarm.jquery.org/

Answer 4

I played with several solutions over the past year, but did not find anything on the Karma football field (previously it was testacular). Try

http://karma-runner.github.com/

Answer 5

You might be able to use a browser-free rhino to run your unit tests on your CI machine. Of course, the drawback here is that it will not find errors specific to the X browser ... but it really beat the installation of 2-3 OSs in your CI box to cover all major platforms ...

But yes, that kind of sucks ... but it can work quite well in a CI script.

Answer 6

I used maven and junit to call a rhino. This is not elegant, but I use it to check the basic services and utility code.

It requires mocking unsupported classes like XHR with Java libraries.

I found that this is the best code in javascript (tests, etc.) and use junit to organize the assembly and bind to CI.

I would like JsTestDriver to be able to do this. Or mocha with a junit reporter.

Answer 7

JS Test Runner is a pretty good solution. It uses PhantomJS and QUnit.