Repeat the mestest test several times

Question

Some of my mstest unit tests help determine the conditions of a multi-threaded race, and therefore they are most useful when running many times in a row, but I want to do this only for specific test runs - not all the time.

Is there a way to configure mstest (preferably in the test list editor) to run the test several times?

Answer 1

I needed to do something similar, so I came up with a solution to this.

This is not easy, but once everything is set up, you can reuse it in projects. I also have this code uploaded to GitHub ( https://github.com/johnkoerner/MSTestLooper ), but in case it goes away at some point, here is how I did it.

First, we create an attribute that we apply to our class to say that it runs all the tests several times. Do it all in a separate assembly, because the DLL needs to live in a special place.

[Serializable] public class TestLooperAttribute : TestClassExtensionAttribute { private static readonly Uri thisGuy = new Uri("urn:TestLooperAttribute"); private string _PropertyName; public string PropertyName { get { return _PropertyName; } set { _PropertyName = value; } } public override Uri ExtensionId { get { return thisGuy; } } public override TestExtensionExecution GetExecution() { return new TestLooperExecution(PropertyName); } } 

Next, we need to create a custom test class execution class:

 class TestLooperExecution : TestExtensionExecution { private string PropertyName; public TestLooperExecution(string PropertyName) { this.PropertyName = PropertyName; } public override ITestMethodInvoker CreateTestMethodInvoker(TestMethodInvokerContext InvokerContext) { return new TestLooperInvoker(InvokerContext, PropertyName); } public override void Dispose() { //TODO: Free, release or reset native resources } public override void Initialize(TestExecution Execution) { //TODO: Wire up event handlers for test events if needed } } 

Finally, we add a custom invoker, where we execute the loop:

 class TestLooperInvoker : ITestMethodInvoker { private TestMethodInvokerContext m_invokerContext; private string PropertyName; public TestLooperInvoker(TestMethodInvokerContext InvokerContext, string PropertyName) { m_invokerContext = InvokerContext; this.PropertyName = PropertyName; } public TestMethodInvokerResult Invoke(params object[] args) { // Our helper results class to aggregate our test results HelperTestResults results = new HelperTestResults(); IEnumerable<object> objects = m_invokerContext.TestContext.Properties[PropertyName] as IEnumerable<object>; foreach (var d in objects) results.AddTestResult(m_invokerContext.InnerInvoker.Invoke(d), new object[1] { d.GetType().ToString()}); var output = results.GetAllResults(); m_invokerContext.TestContext.WriteLine(output.ExtensionResult.ToString()); return output; } } 

The HelperTestResults class simply creates lines for output, you can handle it however you want, and I don't want to include this code because it just makes this post a lot longer.

Compile this into a DLL and then you need to copy it to

 C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PublicAssemblies 

You also need to create a registry entry for the class:

 Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\TestLooperAttribute] "AttributeProvider"="TestLooper.TestLooperAttribute, TestLooper" 

Now that you have done all this, you can finally use the class:

 using System; using Microsoft.VisualStudio.TestTools.UnitTesting; using TestLooper; using System.Collections.Generic; namespace UnitTestSamples { [TestLooper(PropertyName="strings")] public class UnitTest1 { public static List<String> strings = new List<String>(); private TestContext testContextInstance; public TestContext TestContext { get { return testContextInstance; } set { testContextInstance = value; } } [ClassInitialize()] public static void Init(TestContext x) { strings.Add("A"); strings.Add("B"); strings.Add("C"); strings.Add("D"); } [TestInitialize()] public void TestInit() { if (!TestContext.Properties.Contains("strings")) testContextInstance.Properties.Add("strings", strings); } [TestMethod] [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "DataDriven1.csv", "DataDriven1#csv", DataAccessMethod.Sequential)] [DeploymentItem("DataDriven1.csv")] public void TestMethodStrings(string s) { int value1 = Convert.ToInt32(TestContext.DataRow["Col1"]); ; TestContext.WriteLine(String.Format("{0}:{1}", s, value1)); } } } 

Note that our test method accepts a parameter that comes from the test looper. I also show this with a data driven test to show that you can combine the two together to generate large permutations in your datasets.

Answer 2

 [TestMethod()] public void RepetableTest(){ for(int i = 0; i < repeatNumber; i++){ //test code goes here } } 

Answer 3

Consider creating a test to highlight multiple threads. The test list will not allow you to have multiple entries for the same test. However, you can assign a multi-threaded test to your own list and call it only when you want to run this particular test.

Answer 4

I think the answer is no.