Running Fitnesse.NET tests using MSTest

author Richard Fennell calendar Jul 18, 2008 · 5 min read · Uncategorised  ·
Share on: linkedin copy

Update 29 Mar 2010 - See this post for some updated usage notes 

I have been looking at Fitnesse.NET for a while; if you have not come across this testing tool I cannot recommend Gojko Adzic book highly enough; an excellent real world introduction to Fitnesse.NET.

For me, the problem with Fitnesse is that it's WIKI architecture does not lend itself to working within our Team Foundation Server based development model. However, as I am not really looking for a tool to allow Business Analysts to edit a central repository of tests the WIKI was not that important; so I have been looking at a way to store the Fitnesse tests inside a Visual Studio solution, so they are managed like a unit test.

The first step was to get my head round using Fitnesse without a WIKI. This is achieved using FolderRunner,exe that ships as part of Fitnesse.NET. This allows you to define a test as a HTML file as opposed to a WIKI page. Firstly I thought you could just copy the relevant contents.txt page out of the WIKI structure, but this does not work as this file holds the test not as a HTML table, as required, but in the WIKIs own format using | characters to define the table cells. So you have to create the test file containing the Fitnesse tables using your HTML editor of choice. Once you have done this you run the test using the following command line.

..fitnessedotnet2FolderRunner.exe -i userstory1.htm -a MyAssembly.dll -o results

A HTML results file ends up in the results directory and you get a simple results test count on the console.

All this is very good but how can you bolt this into Visual Studio? After a bit of thought I decided that the easiest option was to put the Fitnesse calls within a wrapper unit test. I chose MSTest as I was aiming to run the final solution within TFS Build (but any type if unit testing framework such as nUnit or mbUnit would have done).

Next I decided to make the required calls to run the tests in C#, calling the methods in the Fitnesse DLLs. This was as opposed to trying to shell out and run the FolderRunner.exe.

The easiest way to find the correct calls was to use Reflector to have a look at the code in the FolderRunner.exe (and the other Fit assemblies). After a brief splunk around I found the calls were fairly simple, so this is the process I ended up with to run the tests via MSTest:

  • I started with a solution that contains an application assembly and an second assembly that had all the methods required for the Fit tests (and made sure it all worked via the WIKI and/or command line FolderRunner)

  • Next I added a test project to the solution

  • In the test project I add a reference to the assembly that contains fit test methods and also the fit.dll. I had some problems here that when I ran the test, I got a The location of the file or directory '(path omitted)fit.dll' is not trusted. error. If you get this error follow the process on Don Felker's blog to fix it This blog post seems to have been removed, in summary the post said:

  • If you download a DLL from the Internet, or get it in an email or where ever and you saved it to your disk (including a DLL in a zip too) it has some extra info attached to it called an "AES" file.

  • To fix this annoying issue, go to the DLL in Windows Explorer, right click to view the properties and then click the "Unblock" button at the bottom of the General panel

  • Add an HTML file to the Test Project and in it create the Fit test tables.

  • Set the HTML file properties so it is copied to the output directory.

  • Create a new Test Method to run all the tests in a the HTML file as shown below (obviously you will need to make sure the paths are right to the HTML files and assemblies for your project).

[TestMethod]
public void RunFitnessTests()
{
   fit.Runner.FolderRunner runner = new    fit.Runner.FolderRunner(new fit.Runner.ConsoleReporter());
   var errorCount = runner.Run(new string[] {
       "-i",@"userstory1.htm",
       "-a",@"MyAssembly.dll",
       "-o",@"results"});
    Assert.AreEqual(0, errorCount,runner.Results);
}

  • So now you can make sure this test works using TestDriven.NET to run this test, this checks the test works when the application is built to the standard bindebug directory.
  • However this does not mean it will work with MSTest under the Visual Studio test runner. This is because, by default, the HTML file that holds the test will not be deployed to the TestResults directory (also the actual application assembly under test will not be copied). This is critical, as you have to be careful here, as if the HTML file is missing you get a false success. The Runner.Run method will return 0 as no test failed (as there were none defined) so the assert passes. To get round this you need to add the HTML file (and any other files required) to the list of files to deploy prior to testing. This is done by editing the deployment file list via the menu option Test | Edit Test Run Configuration | Deployment tab. An option to guard against this issue is to add an extra assert to make sure at least some tests were run and we don't have four zeros in the results string, a bit of RegEx will do (this line is not in the sample project by the way)

           Assert.AreEqual(false,Regex.IsMatch(runner.Results, "^0.+?0.+?0.+?0.+?$"),"No tests appear to have been run");

  • Once this is all done you can run the test via the Visual Studio Test menu option and you should see the test run and get a simple pass or fail result with a count of any failures in the error message. (Another possible option would be a make better use of the detailed HTML results file, at present we just use the simple result string for test results details)

So where do we end up?  I think we have a workable solution. We can run Fitnesse tests as easily as any unit test, and as with unit tests the Fitnesse test files are under TFS source control and kept in sync with the application they test. All without the need for the WIKI.