But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Getting code coverage working on Team Build 2010

If you have VS2010 Premium or Ultimate [Professional corrected error in orginal post]  you have code coverage built into the test system. When you look at your test results there is a button to see the code coverage


You would think that there is easy way to use the code coverage in your automated build process using Team Build 2010, well it can done but you have to do a bit of work.

What’s on the build box?

Firstly if your build PC has only an operating system and the Team Build Agent (with or without the Build Controller service) then stop here. This is enough to build many things but not to get code coverage. The only way to get code coverage to work is to have VS2010 Premium or Ultimate also installed on the build box.

Now there is some confusion in blog posts over if you install the Visual Studio 2010 Test Agents do you get code coverage, the answer for our purposes is no. The agents will allow remote code coverage in a Lab Environment via a Test Controller, but they do not provide the bits needs to allow code coverage to be run locally during a build/unit test cycle.

Do I have a .TestSettings file?

Code Coverage is managed using your solution’s .TestSetting file. My project did not have one of these, so I had to ‘add new item’ it via add on a right click in the solution items.

The reason I had no .TestSettings file was because I started with an empty solution and added projects to it, if you start with a project, such as a web application, and let the solution be created for you automatically then there should be a .TestSettings file created.

In the test settings you need to look at the Data & Diagnostics tab and enable code coverage and then press the configure button, this is important.


On the configuration dialog will see a list of your projects and assemblies. In my case initially I only saw the first and the last rows in the graphic below. I selected the first row, the project containing my production code and tried a build.

THIS DID NOT WORK – I had to added the actual production assembly as opposed to the web site project (the middle row shown below). I think this was the key step to getting it going.

The error I got before I did this was Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems.  So if you see this message in the build report check what assemblies are flagged for code coverage.


Does my build definition know about the .TestSettings file?

You now need to make sure that build knows the .TestSettings file exists. Again this should be done automatically when you create a build (if the file exists), but on my build I had to add it manually as I created the file after the build.





So when all this is done you get to see a build with test results and code coverage.


Easy wasn’t it!

Next weeks Agile Yorkshire meeting: Some things about testing that everyone should know, ...... but were afraid to ask, in case somebody told them.

It is Agile Yorkshire time again, it is a real shame that due to the move of the meeting from the 2nd Wednesday to the 2nd Tuesday I really struggle to make the events. Particularly irritating this month as this one look really interesting and the speaker, Ralph Williams, from past evidence always is entertaining. To quote the Agile Yorkshire site the session will..

“The presentation will focus on the techniques that testers use to identify their tests, whether working from a requirements specification or on agile teams.

Agile testing books mostly focus on the agile aspects or the technology so this area often gets glossed over. The main sections would be:

    • Equivalence Classes and Boundary Conditions
    • Decision Tables
    • Classification Trees
    • User Focused Testing

There will be a group exercise looking a how these techniques can be applied to the testing of a well known website.

As a group we will go through the process of identifying the testing that is required and in the process explain various test techniques that might be useful to people back in their day jobs.”

For full details see http://www.agileyorkshire.org/event-announcements/10Aug2010

Running SPDisposeCheck as part of a 2010 CI Build

SPDisposeCheck is a great tool for SharePoint developers to make sure that they are disposing of resources correctly. The problem is it is a bit slow to run, a problem as this will mean developers will tend not to run it as often as they should. A good solution to the problem is to run it as part of the continuous integration process. There is are posts on how to do this via unit tests and as a MSBuild task, but I wanted to use a TFS 2010 style build. Turns out this is reasonably straight forward without the need to write a custom activity.

  • I created a build template based on the Default one.
  • After the compile and before the test step I added a InvokeProcess activity


  • I set the InvokeProcess properties as shown below, the edited settings are
    • Arguments: String.Format(“””{0}””””, outputDirectory) (remember you need the enclosing “ if your path could have spaces in it)
    • Filename: To the location of the SPDisposeCheck.exe file
    • Result: A previously created build variable of type Int32



  • This is done with a simple if check. If there are any errors found I write a build error message and set the TestStatus to failed. You might choose to set the build status to fail or any other flag you wish. The potential problem with my solution is that the TestStatus value could be reset by the tests that follow in the build process, but for a basic example of using the tool this is fine.

So it is easy to added a command line tool to the build. The key reason it is so easy is that SPDisposeCheck returns a number that we can use to see if the test passed or failed. hence we did not need to parse any text or XML results file. I wish more tools did this.

IDD Building a breakfast comment to a become process – now there is a leap

Gil at Typemock has been posting about some ideas we discussed over breakfast at the Typemock Partner conference a while ago, I have been a bit slow at commenting, so I though I better add to the conversation. Though Typemock is an excellent mocking framework, for me basic mocking is not its biggest win. All the ‘classic auto mocking’ of interfaces to speed TDD style working is great, but I can do that with any of the .NET mocking frameworks. All they do is mean I don’t have to write my own test stubs and mocks, so saving me time, which is good but not the keep win I was looking for.

For me there is another way to save much more time and that is to reduce my ‘build, deploy, use’ cycle. In the land of SharePoint this is a significant time saving, or at least has been for us. It has meant that I can replace the build, create WSP, deploy WSP, let SharePoint/IIS restart and then view a web part, with a build and view in ASP.NET page that uses Typemock to fake out all to SharePoint calls. This is what Gil has termed Isolation Driven Development (IDD) Now isn’t a three letter _DD name going a bit far, I am even not sure there enough in it for me to write a book!

That said this is a solid technique which can be applied to any complex environment where developers or testers need a means to mock out significant, costly, or just slow components to ease there daily work process, often enabling some manual testing process, thus making them more productive. If you read the TPS books it mentions a lot how workers should optimise their work space to reduce wasted time the spend moving between machines or roles, this is just such a move.

So if you want to use the technique for Sharepoint have a look at my post, I hope it will save you time whether on SP2007 or 2010, or maybe apply same technique to other technologies.

Today’s DDD South West

Thanks to everyone who turned up for my session at DDD South West, and to the organisers for putting the event on so well.

As my session was basically a 1 hour demo of the testing tools in VS2010 there are no slides for me to upload, but if you have any questions ping me an email. I would say that for a good overview of the subject have a look at the book ‘Professional Application Lifecycle Management with Visual Studio 2010: with Team Foundation Server 2010

Mocking Sharepoint for Testing

In my previous post I talked about using Isolator to mock Sharepoint to aid the speed of the development process. I find this a productive way of working, but it does not really help in the realm of automated testing. You need a way to programmatically explore a webpart, preferably outside of SharePoint to check its correctness.

You could use the methods in my previous post and some form of automated web test, but this does mean you need to spin up a web server of some descriptions (IIS, Cassini etc. and deploy to it) An alternative is look at the Typmock addin Ivonna. This a creates a fake web server to load you page and tools to explore it.

I will describe how to use this technique using the same example as my previous post.

Previously I had placed all the code to fake out SharePoint in the Page_Load event of the test harness page. As I am now trying to write an unit/integration test I think it better to move this into the test itself so I would delete code I placed in the Page_Load event other than any property settings on the actual webpart. I would actually refactor the lines creating the fake URL context and fake SPSite into some helper methods and then call them from my new test. I would then load the page in Ivonna and check it’s values.

I have tried to show this below, using a couple of techniques to show how to get to components in the page.

   1: [TestMethod, Isolated]
   2:      public void LoadWebPage_SpSimpleWebPart_3EntriesInList()
   3:      {
   4:          // Arrange
   5:          TestHelpers.CreateFakeSPSite();
   6:          TestHelpers.CreateFakeURL();
   8:          TestSession session = new TestSession(); //Start each test with this
   9:          WebRequest request = new WebRequest("/SpSimpleTest.aspx"); //Create a WebRequest object
  11:          // Act
  12:          WebResponse response = session.ProcessRequest(request); //Process the request
  14:          // Assert
  15:          //Check the page loaded
  16:          Assert.IsNotNull(response.Page);
  18:          // the the Ivonna extension method to find the control
  19:          var wp = response.Page.FindRecursive<DemoWebParts.SpSimpleWebPart>("wp1");
  20:          Assert.IsNotNull(wp);
  22:          // so we have to use the following structure and dig knowing the format
  23:          // webpart/table/row/cell/control
  24:          var label = ((TableRow)wp.Controls[0].Controls[0]).Cells[1].Controls[0] as Label;
  25:          Assert.IsNotNull(label);
  26:          Assert.AreEqual("http://mockedsite.com",label.Text);
  28:          var list = ((TableRow)wp.Controls[0].Controls[1]).Cells[1].Controls[0] as DropDownList;
  29:          Assert.IsNotNull(list);
  30:          Assert.AreEqual(3, list.Items.Count);
  31:      }

Now I have to say I had high hopes for this technique, but it has not been as useful as I would hope. I suspect that this is due to the rapid changing of the UI design of client’s webparts making these tests too brittle. We have found the ‘mark 1 eyeball’ more appropriate in many cases, as is so often true for UI testing.

However, I do see this as being a great option for smoke testing in long running projects with fairly stable designs.

Mocking Sharepoint for Design with Typemock Isolator

I have found the most productive use of Typemock Isolator with SharePoint is to use it to reduce the time of the F5 cycle (build/deploy/use). If you are using a VPC of some type to do your SharePoint development, as many do, this process can easily take a couple minutes, and these minutes add up.

In my experience webparts usually make fairly simple use of the underlying SharePoint site, by this I mean that they get some data from an SPList(s) or remote data source and render in some way. Or the reverse, they gather data that they drop to an SPLits(s) or remote data source.

So why not remove the requirement for SharePoint during development from the equation? Mock it out with Isolator and an ASP.NET test site.


Consider this scenario…

  • You have a part that lists people name and email address in a combo box
  • This data is stored in an SPList
  • The webpart must also list the URL of the site it is hosted on.

All fairly straight forward, but how to implement the wrapper around it?

  • The first we create an ASP.NET Web Application and an ASP.NET web page in the same solution as the WebParts class library.
  • In this web application reference the webpart project
  • Edit the .ASPX to reference the webpart assembly (line 2) and create an instance on the page (line 11-17). It is best to do this declaratively to avoid any question of the ASP.NET personalisation system.


   1: <%@ Page Language="C#" AutoEventWireup="true" CodeBehind="SpSimpleTest.aspx.cs" Inherits="TestWebSite.SpSimpleTest" %>
   2: <%@ Register Assembly="DemoWebParts" Namespace="DemoWebParts" TagPrefix="wp" %>
   3: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   4: <html xmlns="http://www.w3.org/1999/xhtml">
   5: <head runat="server">
   6:     <title>Untitled Page</title>
   7: </head>
   8: <body>
   9:     <form id="form1" runat="server">
  10:     <div>
  11:         <asp:WebPartManager ID="WebPartManager1" runat="server">
  12:         </asp:WebPartManager>
  13:         <asp:WebPartZone ID="WebPartZone1" runat="server" >
  14:             <ZoneTemplate>
  15:                 <wp:SpSimpleWebPart ID="wp1" runat="server" />
  16:             </ZoneTemplate>
  17:         </asp:WebPartZone>
  18:     </div>
  19:     </form>
  20: </body>
  21: </html>

  • If you browse to this page you will see a null object exception as it loads the web part but it fails when it calls to SharePoint. This is because we have not mocked that yet. Seeing this error does rely on the fact you have a nice big global Try/Catch in the webparts Render() and CreateChildControls() methods. Note a technique I normally recommend but I think vital for this type of component else errors get swallowed by SharePoint


  • Add a reference to the SharePoint assemblies and Typemock Isolator (which of course you need a licensed copy of, or the 30 day demo) in the Test web application
  • In the Page_Load event you now need to add the code to do the faking, I think the comments below explain what is going on. (There is a good argument to refactor much of this into a TestHelper class so it is easy to reuse).
   1: using System;
   2: using System.Collections.Generic;
   3: using System.Linq;
   4: using System.Web;
   5: using System.Web.UI;
   6: using System.Web.UI.WebControls;
   7: using TypeMock.ArrangeActAssert;
   8: using Microsoft.SharePoint;
  10: namespace TestWebSite
  11: {
  12:     public partial class SpSimpleTestWithMockedSP : System.Web.UI.Page
  13:     {
  14:         protected void Page_Load(object sender, EventArgs e)
  15:         {
  17:             // set the name of the list to read data from
  18:             wp1.DataList = "ListName";
  20:             // set the fake return value for the currently running context
  21:             // we can us null as the current parameter as this is what this web page will return
  22:             Isolate.WhenCalled(() => Microsoft.SharePoint.WebControls.SPControl.GetContextSite(null).Url).WillReturn("http://mockedsite.com");
  25:             // create the mock SP Site we are using
  26:             SPSite fakeSite = Isolate.Fake.Instance<SPSite>();
  27:             Isolate.Swap.NextInstance<SPSite>().With(fakeSite);
  29:             // create a fke collection to hold our test data
  30:             var itemCollection = new List<SPListItem>();
  31:             for (int i = 0; i < 3; i++)
  32:             {
  33:                 var fakeItem = Isolate.Fake.Instance<SPListItem>();
  34:                 itemCollection.Add(fakeItem);
  36:                 Isolate.WhenCalled(() => fakeItem["Title"]).WillReturn(string.Format("Title {0}", i));
  37:                 Isolate.WhenCalled(() => fakeItem["Email Address"]).WillReturn(string.Format("email{0}@email.com", i));
  40:             }
  42:             // set what is returned when a call is made for the list
  43:             Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["ListName"].Items).WillReturnCollectionValuesOf(itemCollection);
  44:         }
  45:     }
  46: }
  • Once this code is entered you can browse to the page again and you should see a working webpart that thinks it is talking to a real SharePoint site.


So now you have a way to run you SharePoint dependant webpart outside of SharePoint.This means the F5 cycle is reduced to seconds as opposed to minutes, and debugging is loads easier.

What I find this particularly useful for is sorting CSS and JavaScript issues, where is loads of tiny edits to text files.

THIS DOES NOT MEAN YOU DON’T NEED TO TEST IN SHAREPOINT, but it does means you can avoid much of it. The SharePoint phase become far more a test/QA/UAT process as opposed a development one. This model as a developer productivity enabling one.

In my next post I will talk about Mocking Sharepoint for Test with Typemock Isolator

Speaking at Developer Day South West

I have just heard I will be speaking at Developer Day South West on June the 5th. My subject is Using the new Developer and Test features of VS 2010 to track down and fix bugs, this is basically the same session as I have at our TechDays fringe event yesterday.

Hope to see you there