Silverlight module integration in VS 2010?

Question

I am currently using the Silverlight Unit Test Framework, but I would prefer to run the tests directly in VS2010. I am curious to know which approaches and tools everyone else uses.

I use Silverlight 4 with Prism and the MVVM pattern, and I'm especially interested in the integrated Silverlight Unit Test support in VS 2010, which I can use with ViewModel unit tests. I use dependency injection with Unity, and I write unit tests, mocking calls in my WCF layer, using Moq for Silverlight. I don’t even look at integration tests at the moment, but even in a simple Unit Test, which checks a single ViewModel command, a service request to my mocked service level can take about 50 milliseconds. Therefore, supporting asynchronous tests is important to me.

The problem I am raising here is not related to viewing the testing that I have dealt with using System.Windows.Automation.Peers in the past, and although I have not used it yet, it may now be possible to handle more easily with support in VS 2010 Feature Pack 2 (which seems to be designed to automate / reproduce user interface tests from what I'm compiling).

I should mention that my findings from the products that I have looked at and which have been used so far are as follows:

• Silverlight Unit Test Framework. I am currently using this, and it’s good as much as possible, but its limitations are: (a) it is not integrated with Visual Studio; and (b) if you do not want to run all the tests, you are limited to the raw tag expression filter.
• StatLight is very nice. I am currently using this and have been using it with v0.9 when targeting Silverlight 3 to a previous project. Being a command line tool, it can be integrated with a continuous integration server, which, of course, can cope with another necessary scenario. But this has nothing to do with Visual Studio integration during the development process.
• Unit Test Results viewer for Silverlight (Visual Studio extension in the Visual Studio gallery) - looks promising, but its limitations are: (a) currently it is not possible to find projects that are in the solution folders, and not under the root of the solution; and (b) runs all tests in a given assembly (via StatLight), with no apparent ability to run a specific test or select tests.
• The Einar Ingebrigtsen Silverlight Unit Test Runner for ReSharper, which later became Odin, is ahead of the game (first appeared in 2008), but the limitation is that this project seems to be no longer supported (last updated Apr 2009).
• AgUnit ReSharper plugin ( http://agunit.codeplex.com/ ) - looks great. After downloading the source code and creating the latest (bug fix) version for working with ReSharper 5.1, I was very enthusiastic. But unfortunately, it does not handle asynchronous tests. This is a design limitation with streaming, so it doesn’t matter if you try to use the asynchronous support built into the Silverlight Unit Test Framework (Microsoft.Silverliht.Testing.SilverlightTest base class), or whether you use AutoResetEvent or anything else. This was noted by the coordinator at the CodePlex project discussion forum. This is a massive limitation.
• TestDriven.NET 3.0 seems to have support for Silverlight 4.0 tests at a glance, but the limitation (I suspect) is portability of the Silverlight 4 assembly (i.e. 5 dependent assemblies that migrate between SL4 and .NET 4), of course, when I tried using it with a simple POC, it crashed my VS 2010 instance.

Maybe I missed something here. I wonder if anyone in the community has any better ideas for testing Silverlight modules?

Answer 1

I use Silverlight Unit test, AgUnit and RX with mock IScheduler so my unit tests are single-threaded :)

Answer 2

UPDATE:

I already decided to use StatLight for my continuous integration server, but I was looking for a solution to run Silverlight asynchronous unit tests directly in VS2010 during development.

Inspired by Rob Fonseca-Ensor's suggestion to use Rx (see his separate answer on this page), I looked again at this issue. This led me to find a non-Rx solution to the problem of using AgUnit to run Silverlight asynchronous tests.

I think the solution that I will use, at least for now, is a combination:

• Silverlight Unit Test Framework
• ReSharper 5.1
• AgUnit ReSharper Plugin
• [Mock async pattern]

My ViewModels provides special service classes that provide abstractions of my (auto-generated) WCF service reference classes. To provide access to WCF service methods, my specialized service classes also rely on another common class that wraps the asynchronization pattern. For my unit tests, I already ridiculed my WCF service reference classes and my specialized service classes with Moq - but I did not look at the mockery of my asynchronous template shell class.

So, I decided to mock my asynchronous template wrapper class. The appeal of this was that I thought I could use Rx with a mock IScheduler (as Rob said in his answer) in all my bullying, leaving my real classes without any reference to Rx (which is a requirement because for this project in the place where I work, I need to leave Rx outside of any code that will be deployed in the production environment). However, as soon as I made fun of the wrapper class, I realized that I didn’t even need Rx, and that there was an even simpler solution that I had to really look at earlier. This is pretty trivial — all I really needed to do was make fun of the wrapper class and make sure it called Invoke for callback operations, rather than calling BeginInvoke. This ultimately prevents callbacks that work fine while working in the Silverlight Unit Test Framework in a browser session, from going into a black hole when unit tests run AgUnit in VS2010. (Allowing black hole callbacks when using AgUnit would of course prevent individual checks from being performed properly and could lead to timeouts or, even worse, false positives in the test results.)

In the absence of a better answer to any subsequent answers to this question, it seems to me that this is the easiest way to handle this scenario - letting my asynchronous unit tests run in AgUnit without requiring Rx in my production code.

For those who do not have restrictions on the implementation of code that uses Rx (which is still a DevLabs project) for production, I think Rob's solution is definitely worth a look.

Answer 3

I am the author of the AgUnit plugin.

AgUnit blocks asynchronous unit testing to speed up the test run. Under the covers, the same code is used as the Silverlight Unit Test Framework, but this runner is very slow if you have to process a large number of tests. I saw differences between almost half an hour and a few minutes over several thousand tests.

However, if you want asynchronous testing, this is a very isolated part of AgUnit that does this. I will try to create an assembly with it disabled. In the future it will be a configuration option or attribute, I have not decided yet.

Feel free to contact me with any questions or inquiries.

Answer 4

You read about the recently released Visual Studio 2010 Feature Pack 2 . It is currently available only to MSDN subscribers and only works with some versions of VS2010 (Premium, Ultimate, and Test Pro).

Here's a quick overview: http://blogs.msdn.com/b/amit_chatterjee/archive/2010/11/16/visual-studio-2010-feature-pack-2-released-new-set-of-testing-capabilities. aspx

Answer 5

Usually I create a regular unit test project in VS.NET, add my Silverlight assembly to the links and write unit test classes.

So, everything works out of the box. What is the problem with this solution?