Running Fitnesse.NET tests using MSTest

Update 29 Mar 2010 – See this post for some updated usage notes 

I have been looking at Fitnesse.NET for a while; if you have not come across this testing tool I cannot recommend Gojko Adzic book highly enough; an excellent real world introduction to Fitnesse.NET.

For me, the problem with Fitnesse is that it’s WIKI architecture does not lend itself to working within our Team Foundation Server based development model. However, as I am not really looking for a tool to allow Business Analysts to edit a central repository of tests the WIKI was not that important; so I have been looking at a way to store the Fitnesse tests inside a Visual Studio solution, so they are managed like a unit test.

The first step was to get my head round using Fitnesse without a WIKI. This is achieved using FolderRunner,exe that ships as part of Fitnesse.NET. This allows you to define a test as a HTML file as opposed to a WIKI page. Firstly I thought you could just copy the relevant contents.txt page out of the WIKI structure, but this does not work as this file holds the test not as a HTML table, as required, but in the WIKIs own format using | characters to define the table cells. So you have to create the test file containing the Fitnesse tables using your HTML editor of choice. Once you have done this you run the test using the following command line.

..\fitnesse\dotnet2\FolderRunner.exe -i userstory1.htm -a MyAssembly.dll -o results

A HTML results file ends up in the results directory and you get a simple results test count on the console.

All this is very good but how can you bolt this into Visual Studio? After a bit of thought I decided that the easiest option was to put the Fitnesse calls within a wrapper unit test. I chose MSTest as I was aiming to run the final solution within TFS Build (but any type if unit testing framework such as nUnit or mbUnit would have done).

Next I decided to make the required calls to run the tests in C#, calling the methods in the Fitnesse DLLs. This was as opposed to trying to shell out and run the FolderRunner.exe.

The easiest way to find the correct calls was to use Reflector to have a look at the code in the FolderRunner.exe (and the other Fit assemblies). After a brief splunk around I found the calls were fairly simple, so this is the process I ended up with to run the tests via MSTest:

  • I started with a solution that contains an application assembly and an second assembly that had all the methods required for the Fit tests (and made sure it all worked via the WIKI and/or command line FolderRunner)
  • Next I added a test project to the solution
  • In the test project I add a reference to the assembly that contains fit test methods and also the fit.dll. I had some problems here that when I ran the test, I got a The location of the file or directory ‘(path omitted)\fit.dll’ is not trusted. error. If you get this error follow the process on Don Felker’s blog to fix it This blog post seems to have been removed, in summary the post said:
    • If you download a DLL from the Internet, or get it in an email or where ever and you saved it to your disk (including a DLL in a zip too) it has some extra info attached to it called an “AES” file.
    • To fix this annoying issue, go to the DLL in Windows Explorer, right click to view the properties and then click the “Unblock” button at the bottom of the General panel

  • Add an HTML file to the Test Project and in it create the Fit test tables.
  • Set the HTML file properties so it is copied to the output directory.
  • Create a new Test Method to run all the tests in a the HTML file as shown below (obviously you will need to make sure the paths are right to the HTML files and assemblies for your project).

public void RunFitnessTests()
   fit.Runner.FolderRunner runner = new    fit.Runner.FolderRunner(new fit.Runner.ConsoleReporter());
   var errorCount = runner.Run(new string[] {
    Assert.AreEqual(0, errorCount,runner.Results);

  • So now you can make sure this test works using TestDriven.NET to run this test, this checks the test works when the application is built to the standard bin\debug directory.
  • However this does not mean it will work with MSTest under the Visual Studio test runner. This is because, by default, the HTML file that holds the test will not be deployed to the TestResults directory (also the actual application assembly under test will not be copied). This is critical, as you have to be careful here, as if the HTML file is missing you get a false success. The Runner.Run method will return 0 as no test failed (as there were none defined) so the assert passes. To get round this you need to add the HTML file (and any other files required) to the list of files to deploy prior to testing. This is done by editing the deployment file list via the menu option Test | Edit Test Run Configuration | Deployment tab. An option to guard against this issue is to add an extra assert to make sure at least some tests were run and we don’t have four zeros in the results string, a bit of RegEx will do (this line is not in the sample project by the way)
           Assert.AreEqual(false,Regex.IsMatch(runner.Results, “^0.+?0.+?0.+?0.+?$”),”No tests appear to have been run”);

  • Once this is all done you can run the test via the Visual Studio Test menu option and you should see the test run and get a simple pass or fail result with a count of any failures in the error message. (Another possible option would be a make better use of the detailed HTML results file, at present we just use the simple result string for test results details)

So where do we end up?  I think we have a workable solution. We can run Fitnesse tests as easily as any unit test, and as with unit tests the Fitnesse test files are under TFS source control and kept in sync with the application they test. All without the need for the WIKI.

I have packaged up a sample of this solution and posted to this server.

6 Replies to “Running Fitnesse.NET tests using MSTest”

  1. This looks very promising! I have been looking for an easy way to integrate Fitnesse tests into my projects for a few years, but haven’t had the time to really dig into it. So I haven’t used Fitnesse at all… If you would consider setting up this project at Codeplex, we can all chip in to make this product better. Morten

  2. Glad you found it useful, but I am not sure there is enough to warrent a CodePlex project at this point. The couple of lines of code in the MSTest are all there is really and these could be dropped in any testing framework. Once you get your head round calling the tests like this the pain point becomes setting up MSTest – not fitnesse. I am not sure what the project would be here

  3. What if we needed the WIKI approach is it still possible to use FolderRunner as above?

  4. The problem you have is that FolderRunner wants HTML files not the | separated ones the Wiki uses. You would have to come up with a way to convert the files. However, could you not just run the command line TestRunner that acutally uses the Wiki as it data source? I was not using it as I wanted the files under TFS source not in thw Wiki

  5. Hi Richard, This is really cool. Im trying to integrate FitNesse with VS solution in the same manner. Thanks for the informative post! I ran into the same problem, The location of the file or directory ‘(path omitted)\fit.dll’ is not trusted. error and the link to rectify the problem is down. Could do explain how you solved it?

  6. Radhika It does not help when the linked post is removed, good old google cache! I have edited this post to contain details of the fix for the ‘(path omitted)\fit.dll’ is not trusted. error

Comments are closed.