Exclusion of tests from the tfs assembly

Question

I want to exclude some tests from my continuous integration assembly, but I have not found a way to do this.

One of the things I tried was to set the priority of these tests to -2, and then on the assembly that I set Minimum Test Priority = -1, but it still runs these tests.

Any help would be greatly appreciated.

Answer 1

Instead of using the "test lists" that have been described, you should use the "Test Category" method. Test lists and VSMDI functionality are actually outdated in Visual Studio 2010, and Microsoft may completely remove this feature in a future version of Visual Studio.

If you want more information on how to use test categories, especially in the automatic build process, check out this blog post: http://www.edsquared.com/2009/09/25/Test+Categories+And+Running + A + Subset + Of + Tests + In + Team + Foundation + Server + 2010.aspx

You can also exclude test categories from work by specifying a symbol ! (exclamation mark) in front of the category name to further define your filter.

Answer 2

If you use MSTest, you can create a Test List for the tests you need with continuous integration.

Answer 3

In MSTest, you can simply create two test projects (assemblies) and specify only one of the assembly configurations that will be used for testing. In MSBuild, that was the way to go. For the new WF-based build definitions, I currently don't have a sample:

 <ItemGroup> <!-- TEST ARGUMENTS If the RunTest property is set to true then the following test arguments will be used to run tests. Tests can be run by specifying one or more test lists and/or one or more test containers. To run tests using test lists, add MetaDataFile items and associated TestLists here. Paths can be server paths or local paths, but server paths relative to the location of this file are highly recommended: <MetaDataFile Include="$(BuildProjectFolderPath)/HelloWorld/HelloWorld.vsmdi"> <TestList>BVT1;BVT2</TestList> </MetaDataFile> To run tests using test containers, add TestContainer items here: <TestContainer Include="$(OutDir)\AutomatedBuildTests.dll" /> <TestContainer Include="$(SolutionRoot)\TestProject\WebTest1.webtest" /> <TestContainer Include="$(SolutionRoot)\TestProject\LoadTest1.loadtest" /> Use %2a instead of * and %3f instead of ? to prevent expansion before test assemblies are built --> </ItemGroup> <PropertyGroup> <RunConfigFile>$(SolutionRoot)\LocalTestRun.testrunconfig</RunConfigFile> </PropertyGroup> 

Tip. To use the definition of a common assembly, we call all our test projects "AutomatedBuildTests", i.e. there is no difference in decisions. Thus, an assembly definition can be included in any existing assembly definition (or even be normal) that always performs the correct set of tests. It would be a daunting task to provide an if-if check to allow the assembly definition to run tests only when a test assembly is present. We do not use this to get assembly errors if a test assembly is not found, since we absolutely want to test all these assemblies that use this definition.

Answer 4

My preference would be as above using a test list, but some people have released merging / editing vsmdi files ... We end up with separate solutions and use pattern matching to run all the tests in the corresponding DLL.

Answer 5

In Visual Studio 2012 and later, you can customize the build definition using the Test case filter parameter.

This parameter is part of the definition of your assembly. Open the assembly definition and go to the Process tab. In section 3. Test you can define several test sources. For each test source, you can specify a Test case filter .

For more information, see this MSDN article: Running Custom Unit Tests in VS 2012 RC Using TestCaseFilter

I copied the supported statements and some examples from this article:

Operators supported in RC:

1. = (equal)

2.! = (Not equal)

3. ~ (contains or substring only for string values)

4. Amplifier & (AND)

5. | (Or)

6. () (paranthesis for grouping)

An express can be created using these operators as any valid logical condition. and (and) has a higher priority over | (or) when evaluating an expression.

eg.
"TestCategory = NAR | Priority = 1" "Owner = Vikram &! TestCategory = UI" "FullyQualifiedName ~ NameSpace.Class"
"(TestCategory! = UI & (Priority = 1 | Priority = 2)) | (TestCategory = UI & Priority = 1)"

Another possibility would be to have some test sources in one build definition in some (i.e. more or less) test sources in other build definitions.