SpecFlow - Parallel Tests

Question

I use tests using SpecFlow, which have nothing to do with each other. Is there a configuration option for SpecFlow that allows parallel testing? I use the VS10 and MSTest runner, which supports running up to 5 parallel unit tests, as they claim in the documentation.

Thanks max.yz

Answer 1

There is a new tool called SpecRun , which was recently released by the creators of SpecFlow. SpecRun will let you run these tests in parallel. If you use the SpecRun.Nunit package with it, you can run NUnit tests in parallel. We use SpecRun on our CI server to run tests in parallel, but the developers use whatever they want, their choice.

Changing the scope of testing can be devastating. Since all of our tests were in NUnit for starters, we just added a new SpecRun runner and didn't change anything else. Very simple and transparent to developers. And since it is available on NuGet, it was very easy to install.

Answer 2

I moved from MSTest to MbUnit to achieve this. You can achieve parallelism at the test device level with MbUnit using ParallelizableAttribute. However, since test fixtures are generated from .Feature Gherkin files, I had to grab the SpecFlow source code and change the MbUnitTestGeneratorProvider class in the TechTalk.SpecFlow.Generator project to output ParallelizableAttribute. So you get something like this:

public class MbUnitTestGeneratorProvider : IUnitTestGeneratorProvider { private const string TESTFIXTURE_ATTR = "MbUnit.Framework.TestFixtureAttribute"; private const string PARALLELIZABLE_ATTR = "MbUnit.Framework.ParallelizableAttribute"; private const string TEST_ATTR = "MbUnit.Framework.TestAttribute"; private const string ROWTEST_ATTR = "MbUnit.Framework.RowTestAttribute"; private const string ROW_ATTR = "MbUnit.Framework.RowAttribute"; private const string CATEGORY_ATTR = "MbUnit.Framework.CategoryAttribute"; private const string TESTSETUP_ATTR = "MbUnit.Framework.SetUpAttribute"; private const string TESTFIXTURESETUP_ATTR = "MbUnit.Framework.FixtureSetUpAttribute"; private const string TESTFIXTURETEARDOWN_ATTR = "MbUnit.Framework.FixtureTearDownAttribute"; private const string TESTTEARDOWN_ATTR = "MbUnit.Framework.TearDownAttribute"; private const string IGNORE_ATTR = "MbUnit.Framework.IgnoreAttribute"; private const string DESCRIPTION_ATTR = "MbUnit.Framework.DescriptionAttribute"; public bool SupportsRowTests { get { return true; } } public void SetTestFixture(CodeTypeDeclaration typeDeclaration, string title, string description) { typeDeclaration.CustomAttributes.Add( new CodeAttributeDeclaration( new CodeTypeReference(TESTFIXTURE_ATTR))); typeDeclaration.CustomAttributes.Add( new CodeAttributeDeclaration( new CodeTypeReference(PARALLELIZABLE_ATTR))); SetDescription(typeDeclaration.CustomAttributes, title); } 

If you compile this and use it, you will get parallelizable test equipment:

 [System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "1.6.1.0")] [System.Runtime.CompilerServices.CompilerGeneratedAttribute()] [MbUnit.Framework.TestFixtureAttribute()] [MbUnit.Framework.ParallelizableAttribute()] [MbUnit.Framework.DescriptionAttribute("Test")] public partial class TestFeature { 

The only problem with this is that you will need to make sure that the test devices do not conflict with each other. That is, a test from a single device adds or modifies a database row that breaks a test that runs at the same time. There are ways around this, but this is probably beyond your original question.

Alex

Answer 3

I created a solution that generates a nant build file that uses nunit in a custom parallel nant task:

https://github.com/MartyIce/SpecflowParallelizer

Due to the way my test tests were written, I get problems with backend concurrency, so for me it was not successful (but still), but hopefully this will work for someone else.

Answer 4

There is an option in the MSTest.testsettings file for the test project. By default, a test runner will run only one test at a time, changing the parallelTestCount Execute node attribute to 0, it will work on as many threads as avalable (for some reason, a maximum of 5 is limited)

Just right-click the .teststtings file and open it; select the XML editor and release.

You should not run any tests with a coded user interface or configure any datacollectors for this.

See this article for a more detailed explanation .