Lighthouse – Silverlight Unit Test Runner Project released
Unit Testing in Silverlight? But you need Web Browser to do that!
No you do not! Ok maybe you do. But not in a direct way. I will clarify that later. Lets first go back in time to the root of the story:
Since the initial release of Silverlight i was really annoyed by the fact that Unit Testing had very slim support.
Even later when Silverlight Unit Testing Framework was introduced by Microsoft things became little better but still it was far from good.
You could create Unit Tests, but you had to start your Visual Studio in order to run them – eventually looking at the results of tests in your favorite Web Browser.
That ruled out one of the most important facets of Test Driven Development – running your Unit Test on Continuous Integration server, so that you can easily see when any of your Unit Tests are broken – in other words treating your Unit Tests equally as all your code, like you should.
Not to mention that there was no way to run Unit Tests from Reshaper! (oh boy don’t even let me get started on that one).
Can feeling of anger be a motivation for good deeds?
Very likely. I started by investigating what open source projects were available to fill these Silverlight TDD gaps.
And i did found few of them. But they were either too complex or simply did not work properly and failed to run tests from one reason or the other.
This is how i decided to create project Lighthouse – probably in same way how many other open source projects are born: to scratch an itch of single developer, for the benefit of the community 🙂
What is it for? What’s the Roadmap?
Lighthouse is envisioned as project that will host multiple tools that will allow Silverlight developers to do Test Driven Development much easier then before.
First tool in this project Silverlight Command Line Unit Test Runner has just been released on Codeplex website.
Lighthouse Command Line Unit Test Runner is console application that runs given Silverlight Unit Testing Framework Unit Tests and collects their results and saves them in known NUnit XML Unit Tests results file format on your hard-drive.
What can i expect?
The rule of the thumb goes like this:
YOUR SILVERLIGHT UNIT TESTS SHOULD BEHAVE EXACTLY THE SAME WHEN YOU RUN THEM WITH LIGHTHOUSE AS IF YOU ARE RUNNING THEM FROM VISUAL STUDIO OR WEB BROWSER.
In other words: If they failed in browser, they will fail when you run them via Lighthouse in completely same way (same timing, same exception etc) – and vice versa.
You can use Lighthouse to run tests in XAP packaged application or you can use it to run tests directly specifying your Silverlight DLL’s – it is very flexible.
Currently Lighthouse saves results in NUnit XML Unit Tests results file format so that means you can use it to run your Unit Tests in ANY Continuous Integration server that supports this format – CCNet, Team City you name it.
In future we will maybe add more output XML formats, but for start this should suffice.
Lighthouse Silverlight Command Line Unit Tests runner supports variety of command line switches so you can tweak it to work how you need it.
You can specify Tag Filter to run only certain test cases.
You can specify timeouts, output log file, output directory etc.
Make sure you visit the Lighthouse Command Line Test Runner Documentation page to get familiar with all the options.
In the future we are planning to develop Resharper plugin to run Silverlight Unit Tests, but you will here more on that when its ready.
And how does all this work?
Its much simpler to explain then it was to make it work:
In its essence, Lighthouse analyzes your XAP/Assemblies and modifies them in a way so that it can control the Unit Test execution. It then fires up a web browser instance in the background and start this modified Silverlight application and collects the results of all Unit Tests. Results are saved in XML file and everyone go home and live happily ever after.
Until future releases I’m asking all interested Silverlight developers to use Lighthouse and send us feedback or even join the Lighthouse team to make it better.
For the end check out this short video where Lighthouse Command Line Runner executes Silverlight Unit Tests from Sterling project:
17 thoughts on “Lighthouse – Silverlight Unit Test Runner Project released”
How is that different from Statlight? http://statlight.codeplex.com/
Why not join efforts to build one good SL testing tool instead of two 80% there?
Lighthouse is very different than Statlight because its more focused and simpler.
Lighthouse aims just Silverlight Unit Testing Framework while Statlight aims other Unit Testing Frameworks (Nunit etc)
As i stated in my blog post i just wanted simple unit test runner that works out of the box as i expect it to.
All other unit test runners for Silverlight failed to do so in my opinion so this is how i decided to build Lighthouse.
My biggest issue (and why I ultimately don’t use Statlight) is because my CI server runs as a service with desktop interactivity disabled (which I unfortunately do not have control over) which prevents the Statlight test runner from ever being loaded.
Is this possible with Lighthouse? Or is this just the nature of the beast?
Thanks, and this looks great.
I was able to run Lighthouse with CCNet and Team City running as services without setting up desktop interactivity so I guess it must work.
I suggest to go to http://lighthouse.codeplex.com and download latest installer and try it out.
Please do let me know how it worked out for you.
Thanks for your feedback!
it is very very cool project, because I spent almost 1 year to find out how to run my Silverlight unit tests on TeamCity. I tried Statlight, but it failed, I tried Odin, and even parsing browser results with windows powershell.
So, I tried Lighthouse and it worked!
But for now I have a question:
How do you get test-suite name (from test results file)?
It seems to me that it is hardcoded now (test-suite name=”Prj”) because I see name that does not correspond to the real one.
thanks for your feedback, this is exatly why i created Lighthouse, just so it works without too much hassle to set it up.
You are right, test-suite is hard-coded, i was not sure what to put there so i left it to be done later.
What are your suggestions, what should be there? Namespace of the project?
in my current project nunit tests results in TeamCity look like this:
TestClassName.TestCaseName (My.dll: Name.Space.Path)
So, it could be at least test class name where test is stored.
I just created ticket for this:
i will try to fix this as soon as possible.
thanks again for taking time to report this i appreciate it!
new version of Lighthouse is released that included the fix you requested.
have fun 🙂
When I try to execute the command : Lighthouse.exe –m:Dll -tests:”E:\Target\MyTests.dll” -references:”E:\Target\Code*.dll” E:\tmp\results.xml ,
I get the error “The given path’s format is not supported”.
Can you please tell me why this is happening
please use this link:
and post description how to reproduce this error, then i can try to help you.
I’m looking for a unit test runner for Silverlight and I just want to know if this project is alive, especially if it is gonna be compatible with SL 5? When looking at the codeplex site it looks like it’s asleep.
currently i don’t have plans to continue supporting this project since i moved to other things.
However if you want to get involved let me know i can add you to the list of contributors.
Also one user already found solution for SL 5 problem take a look here:
Here’s the thing I did yesterday for my client using VS2012:
– created SL project with some MUT
– Added standard MS Unit Test project
– Referenced SL DLL to the unit test project (referenced DLL turned yellow in UI)
– wrote a unit test calling method from that DLL
– run it using R# test runner
– Test passed.
No clue if it helps with CI story, but I guess that too is much easier to be established without helpers
Yes it does work for some cases, for others it may not.
Lighthouse creates full, real Silverlight environment for tests inside browser window but just reports results to the ‘outside world’ so its
more complete solution.
But in any case, Silverlight seems to be history so it does not matter anyhow 🙂