Database to import NUnit results?

Question

I have a large NUnit test suite; I need to import the results from a specific launch into the database, and then characterize the set of results and present them to users (email for test failures, web presentation to study the results). I need to track multiple starts over time as well (to report errors over time, etc.).

XML will be XML created by the nunit console. I would like to import XML with a minimum of fuss into some database, which can then be used to save and present the results. We will have a number of custom categories that we will also need to sort.

Does anyone know a database schema that can handle the import of a type of data that can be customized for our individual needs? This type of problem seems like it should be general, and therefore there must be a general solution for it, but I cannot find it. If someone has already implemented such a solution, the advice will also be appreciated.

Answer 1

It seems to me that you are actually after the build server, for example CruiseControl.NET or TeamCity .

Get the build server to run the tests, and it performs the task of telling people what failed and why.

I recommend TeamCity, as it is several orders of magnitude easier to set up.

Answer 2

I am here to solve the same problem. We are currently leaning towards writing XSLT to convert XML results to insert statements. Then run the resulting insert statement file using the SQL command-line interpreter. Ideally, I would rather have a NUnit add-in / extension that handles all this for me. Unfortunately, I could not find him.

Answer 3

To build IainMH's answer, you can take a look at using Trac with BITTEN , an open source system that can run n-unit tests and report results. I am currently using it for this exact functionality.

Answer 4

When using MS SQL, you can import all XML files into a common data type column [xml]. At the same time, xpaths, search, and conversions can be performed.

Answer 5

Another alternative to CruiseControl or TeamCity is Atlassians Bamboo if you are attached to cash. I am a big fan of my software because it is easy to use and they have a deal where you can get bamboo for 10 bucks.

Answer 6

We hoped to avoid this, but we created a database schema from an XML NUnit result schema; however, it is slightly inadequate, as NUnit does some (inaccurate and strange) processing to determine some critical statistics (for example, "ignored" or "does not work").

We still hope to find a schema / process that is NOT a complete CIT build system that allows us to set up a database to import the results, but we are currently using a manual database, which we will need to do a lot of customization to get the required reporting.

Answer 7

Why do you need results in a database? Who will use them? The number of failures cannot be large. If this is (repeatedly), your development process is wrong. Correct the process. Eliminate waste (one of the lean principles), do not collect it.

Take smaller steps (shorter iterations, continuous assembly), eliminate dependencies.

This is usually not done because projects that have such problems are not delivered, but canceled (in the end).

[edit] Michael, tracking nunit errors for a longer time provides a null value. You need a short feedback loop. Fix problems now. If you wait until you have a lot of problems, you will be amazed at the noise.

Good problem tracking is done at the right level (the maximum possible abstraction). Definitely not a unit test.