But it works on my PC!

Running Fitnesse.NET tests using MSTest

Richard — Fri, 18 Jul 2008 22:00:00 GMT

I have been looking at Fitnesse.NET for a while; if you have not come across this testing tool I cannot recommend Gojko Adzic book highly enough; an excellent real world introduction to Fitnesse.NET.

For me, the problem with Fitnesse is that it's WIKI architecture does not lend itself to working within our Team Foundation Server based development model. However, as I am not really looking for a tool to allow Business Analysts to edit a central repository of tests the WIKI was not that important; so I have been looking at a way to store the Fitnesse tests inside a Visual Studio solution, so they are managed like a unit test.

The first step was to get my head round using Fitnesse without a WIKI. This is achieved using FolderRunner,exe that ships as part of Fitnesse.NET. This allows you to define a test as a HTML file as opposed to a WIKI page. Firstly I thought you could just copy the relevant contents.txt page out of the WIKI structure, but this does not work as this file holds the test not as a HTML table, as required, but in the WIKIs own format using | characters to define the table cells. So you have to create the test file containing the Fitnesse tables using your HTML editor of choice. Once you have done this you run the test using the following command line.

..\fitnesse\dotnet2\FolderRunner.exe -i userstory1.htm -a MyAssembly.dll -o results

A HTML results file ends up in the results directory and you get a simple results test count on the console.

All this is very good but how can you bolt this into Visual Studio? After a bit of thought I decided that the easiest option was to put the Fitnesse calls within a wrapper unit test. I chose MSTest as I was aiming to run the final solution within TFS Build (but any type if unit testing framework such as nUnit or mbUnit would have done).

Next I decided to make the required calls to run the tests in C#, calling the methods in the Fitnesse DLLs. This was as opposed to trying to shell out and run the FolderRunner.exe.

The easiest way to find the correct calls was to use Reflector to have a look at the code in the FolderRunner.exe (and the other Fit assemblies). After a brief splunk around I found the calls were fairly simple, so this is the process I ended up with to run the tests via MSTest:

I started with a solution that contains an application assembly and an second assembly that had all the methods required for the Fit tests (and made sure it all worked via the WIKI and/or command line FolderRunner)
Next I added a test project to the solution
In the test project I add a reference to the assembly that contains fit test methods and also the fit.dll (I had some problems here that when I ran the test, I got a The location of the file or directory '(path omitted)\fit.dll' is not trusted. error. If you get this error follow the process on Don Felker's blog to fix it)
Add an HTML file to the Test Project and in it create the Fit test tables.
Set the HTML file properties so it is copied to the output directory.
Create a new Test Method to run all the tests in a the HTML file as shown below (obviously you will need to make sure the paths are right to the HTML files and assemblies for your project).

[TestMethod]
public void RunFitnessTests()
{
   fit.Runner.FolderRunner runner = new    fit.Runner.FolderRunner(new fit.Runner.ConsoleReporter());
   var errorCount = runner.Run(new string[] {
       "-i",@"userstory1.htm",
       "-a",@"MyAssembly.dll",
       "-o",@"results"});
    Assert.AreEqual(0, errorCount,runner.Results);
}

So now you can make sure this test works using TestDriven.NET to run this test, this checks the test works when the application is built to the standard bin\debug directory.
However this does not mean it will work with MSTest under the Visual Studio test runner. This is because, by default, the HTML file that holds the test will not be deployed to the TestResults directory (also the actual application assembly under test will not be copied). This is critical, as you have to be careful here, as if the HTML file is missing you get a false success. The Runner.Run method will return 0 as no test failed (as there were none defined) so the assert passes. To get round this you need to add the HTML file (and any other files required) to the list of files to deploy prior to testing. This is done by editing the deployment file list via the menu option Test | Edit Test Run Configuration | Deployment tab. An option to guard against this issue is to add an extra assert to make sure at least some tests were run and we don't have four zeros in the results string, a bit of RegEx will do (this line is not in the sample project by the way)

Assert.AreEqual(false,Regex.IsMatch(runner.Results, "^0.+?0.+?0.+?0.+?$"),"No tests appear to have been run");

Once this is all done you can run the test via the Visual Studio Test menu option and you should see the test run and get a simple pass or fail result with a count of any failures in the error message. (Another possible option would be a make better use of the detailed HTML results file, at present we just use the simple result string for test results details)

So where do we end up? I think we have a workable solution. We can run Fitnesse tests as easily as any unit test, and as with unit tests the Fitnesse test files are under TFS source control and kept in sync with the application they test. All without the need for the WIKI.

I have packaged up a sample of this solution and posted to this server.

Registration opened for SQLBits III (Cubed)

Richard — Fri, 18 Jul 2008 08:59:41 GMT

I see that the registration (and hence session voting) has opened for the free community conference SQLBits III,

I can't make it as it is the same day as Alt.Net. However, I really do recommend this event if you ever work with data (some of you do that don't you?)

New Team System Power Tools

Richard — Thu, 17 Jul 2008 10:19:30 GMT

The July edition of the Team System power tools have just been released.

To pick out just one of a great set of tools - have a look at the Alert Editor, a really easy way to wire up external events handler to do things when TFS events occur.

Leeds SQL Usergroup

Richard — Thu, 17 Jul 2008 09:32:12 GMT

Thanks to everyone one who attended my session at the Leeds SQL User group on Visual Studio Database Edition and to UPCO for hosting the event.

The slide stack is available on the Black Marble web site

And as predicted....

Richard — Sat, 12 Jul 2008 08:49:58 GMT

The Alt.net summer conference is now full after about 24 hours and that is with a larger capacity venue than the last one.

Told it would be fast

I've registered have you?

Richard — Fri, 11 Jul 2008 08:07:23 GMT

Registration has opened for the Alt.UK Summer conference. If the first UK Alt.net conference event is anything to go by it will fill up fast, so don't hang around if you are interested

Publishing a ASP.NET WCF service encryption error

Richard — Tue, 08 Jul 2008 09:30:52 GMT

I was trying to publish a .NET 3.5 WCF service to a network share e.g.\\myserver\share\folder_to_holdservice and got the error "The specified file could not be encrypted" for all the files.

I changed to a publish to the local disk it published fine, so what caused that?

As part of some security impact modelling I have been doing development with my local source directories encrypted using standard Vista security (FYI does not seem to cause any significant performance impact)

So when I copied the published files to the network share manually I got an message 'do I want to copy without encryption' which I answered yes to and all was OK. Shame VS publish method does not have a away to answer this question

Next XP Club meeting

Richard — Mon, 07 Jul 2008 13:26:10 GMT

This months meeting of the XPClub is on Wednesday. It is a general group chat about refactoring, branching and merging.

This is a free event, usually with a few free beers, at the usual time & location, 7pm at the Victoria Hotel in Leeds, so hope to see you there

Bug tracking with TFS

Richard — Sun, 06 Jul 2008 22:37:25 GMT

I have posted in the past about my efforts to write a user facing bug tracking interface for TFS to integrate with our SharePoint based customer portal. I have had some mixed success, but the end point is that I am just not happy with what I have written.

Historically we have used our own home grown call tracking system (started as an Access DB, went via VB6 to ASP then ASP.NET, now is web service based) which our clients know (and love?). This give a far richer audit trail for the actions performed on a support call than is possible with a work item in TFS. In the end this simple fact is what has forced me to conclude that TFS work items are not the thing to expose to end user/help desk staff for bug tracking.

My new plan is to add a new feature to our existing call tracking system (oh and of course port it into a SharePoint webpart based UI) that allows a call to be escalated into TFS when it becomes change request or requires a code bug fix. This means all the initial triage can be handed by the support desk in a their call tracking system and TFS only gains a work item when it is a task for the development team, a product backlog item in Scrum terminology.

This seems a more sensible approach, much like the Snagit add-in for TFS, I will report back as to how it goes.

DDD7 Session Submissions

Richard — Thu, 03 Jul 2008 12:22:57 GMT

It really good to see so many submissions for the DDD7, and many new names, always a good sign of a the community working well.

I just got round to putting one in myself on integrating testing into MSBuild.

There is still time for more though. So have a think and put in a session proposal - trust me it is great fun.

Wow I'm an MVP

Richard — Wed, 02 Jul 2008 08:42:48 GMT

Found out last night that I have just been made an MVP for Team System, is that cool or what!

TFS Team Build compile details

Richard — Mon, 23 Jun 2008 20:31:34 GMT

Martin Woodward posted a few weeks ago about his build bunny. Now this is something I had tried a while ago for DDD4, but hit the same problem Martin had that the old Nabaztag API was too slow and messages could take hours to arrive, making it useless in the real world. Inspired by Martin and the new faster API I have been working on a new Team Build status monitor for the office.

Getting most of it going is straight forward as Martin said, the TFS API event model is easy to use. However I did have a problem getting the details of the build. I could see if it built or not, but I could not get any count of warning, code analysis errors or test results etc.

It seems the problem is that when the BuildQueue_StatusChanged event fires after a build completes it does not have all the details in the IBuildDetail object you would expect. However, if you re-query the TFS server you can get the information.

I added the following to the end of the BuildQueue_StatusChanged event handler:

// first get the handle to the queue (you might have this stored already)
TeamFoundationServer tfs = new TeamFoundationServer("http://myserver:8080");
IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
IQueuedBuildsView buildQueue = buildServer.CreateQueuedBuildsView("MyProject");

// Now re query using the project build URL to get the number of errors and warning
// the array of string list what details we want
IBuildDetail buildDetails = buildServer.GetBuild(buildQueue.QueuedBuilds
         [lastItem].Build.Uri, new string[] {
              TeamFoundation.Build.Common.InformationTypes.ConfigurationSummary,
              TeamFoundation.Build.Common.InformationTypes.TestSummary,
              TeamFoundation.Build.Common.InformationTypes.CodeCoverageSummary,
              TeamFoundation.Build.Common.InformationTypes.CompilationSummary},
              QueryOptions.None);

// Now use the new IBuildDetail object to update the screen associated display
// user control
this.display.UpdateStatus(buildDetails);

Now in my display usercontrol all I need to go is to extract compiler summary details using a standard TFS InformationNodeConverter (and you could do the same for test or code coverage).

var buildSummaries = InformationNodeConverters.GetConfigurationSummaries(build Details);
if (buildSummaries.Count > 0)
{
this.lblCompileReport.Text = "[Errors:" + buildSummaries[0].TotalCompilationErrors +
"] [Warnings:" + buildSummaries[0].TotalCompilationWarnings +
"] [Static Analysis Errors:" + buildSummaries[0].TotalStaticAnalysisErrors +
"] [Static Analysis Warnings:" + buildSummaries[0].TotalStaticAnalysisWarnings +"]";
}

Speaking at Leeds SQL User group

Richard — Sat, 21 Jun 2008 21:38:17 GMT

For those who are interested, I am speaking on Visual Studio for Database Professionals at the Leeds SQL user group on the 16th July. This is a free event, for more details see the event web site.

CA0055 error in FXCop

Richard — Fri, 20 Jun 2008 16:29:02 GMT

I have been setting up a new Team Build server today. All our projects are being set to do code analysis (FXCop) after the build. For one project this worked on the developer PC but failed on the build machine.

The CA0055 error means 'file not found' or 'could not load' the assembly to be analysed. Firstly I suspected there was a problem with path names being over 256 characters (both the assemblies and solutions names were long) which can be a problem MSBuild, but this was not the case.

In the end I looked in the *.CodeAnalysisLog.xml file in the build directory. This gave the inner exception and I found it was a missing DLL that was referenced by a DLL the main DLL referenced - so two hops away! I added this DLL to the project references to make sure it was in the build directory and all was OK.

TIP: read the detailed FXCop log file - it shows far more than the summary in the IDE

XPClub Meeting about Opera

Richard — Thu, 12 Jun 2008 11:40:01 GMT

Good session at the Yorkshire Extreme Programming Club last night. Chris Mills of Opera spoke on the mobile web.

If this, or the general issue of standards in web development are of interest http://dev.opera.com is well worth a look.