Testing SharePoint Workflows using TypeMock Isolator (Part 3)

Updated 12 June 2009 – I have been having problems using this technique of Typemock with Sharepoint Workflows, the workflows keep unexpectedly idling as opposed to activating. If you suffer similar problem please check for later posts as to any solutions I find. 

Now I can test a basic workflow it soon becomes obvious that you could end up with many tests for a single workflow, as a workflow can have any number of criteria that could cause branching to occur. Maybe a sensible way to write the tests is using Fit/Fitness to provide the test cases in tabular form?

So to this end I have added a bit more code to my Typemock/Sharepoint test project (after installing the Fit libraries as detailed in my previous posts). I now have a single test that loads a set of test criteria from an HTML file (my previous posts discuss why I am using HTML files as opposed to the Fit Wiki).

   1: [TestMethod]

   2:      public void WorkFlow1SwitchOnTitle_DataViaFitnesse_Success()

   3:      {

   4:          fit.Runner.FolderRunner runner = new fit.Runner.FolderRunner(new fit.Runner.ConsoleReporter());

   5:          var errorCount = runner.Run(new string[] {

   6:                  "-i",@"WorkflowTestCases.htm", // the htm file that holds the test

   7:                  "-a",@"TestProject.dll",  //we have the fit facade in this assembly

   8:                  "-o",@"results"}); // the directory the results are dumped into as HTML

   9:          // fit can fail silently giving no failures as no test are run, so check for exceptions

  10:          Assert.AreEqual(false, Regex.IsMatch(runner.Results, "^0.+?0.+?0.+?0.+?$"), "No tests appear to have been run");

  11:          // look for expected errors

  12:          Assert.AreEqual(0, errorCount, runner.Results);

  13:          

  14:      }

I then have an HTML file that contains the test cases (remember to make sure that this file is deployed to the output directory)

import
TestProject
Workflow Fit Tests
Upload Document With Title Document Approved?
ABC True
XYZ False
abc True

 

   1: <HTML><HEAD>

   2:     <body>

   3: <table border="1" cellspacing="0">

   4: <tr><td>import</td>

   5: </tr>

   6: <tr><td>TestProject</td>

   7: </tr>

   8: </table>

   9:     

  10: <table border="1" cellspacing="0">

  11: <tr><td colspan="2">Workflow Fit Tests</td>

  12: </tr>

  13: <tr><td>Upload Document With Title </td>

  14: <td>Document Approved?</td>

  15: </tr>

  16: <tr><td>ABC</td>

  17: <td>True</td>

  18: </tr>

  19: <tr><td>XYZ</td>

  20: <td>False</td>

  21: </tr>

  22: <tr><td>abc</td>

  23: <td>True</td>

  24: </tr>

  25: </table>

  26:  

  27:     </body>

  28: </html>

 

Finally we need to create the facade class that wrapper the workflow function for Fit to call. In this sample I just popped the class in the test project for simplicity. Notice it is this facade class that contains all the Typemock bits, also that I make use of the helper class I created in my previous post to actually run the workflow.

   1: using System;

   2: using TypeMock.ArrangeActAssert;

   3: using Microsoft.SharePoint.Workflow;

   4:  

   5: namespace TestProject

   6: {

   7:     public class WorkflowFitTests : fit.ColumnFixture

   8:     {

   9:         public string UploadDocumentWithTitle;

  10:  

  11:         public bool DocumentApproved()

  12:         {

  13:  

  14:             var fakeProperties = Isolate.Fake.Instance<SPWorkflowActivationProperties>();

  15:             var fakeItem = fakeProperties.Item;

  16:             Isolate.WhenCalled(() => fakeItem.Title).WillReturn(this.UploadDocumentWithTitle);

  17:  

  18:             // Act

  19:             TypemockWorkflowTests.WorkflowRunner(typeof(SharePointWorkflow.Workflow1), fakeProperties);

  20:  

  21:             // Assert, if a document is approved must call the following two line

  22:             try

  23:             {

  24:                 Isolate.Verify.WasCalledWithExactArguments(() => fakeItem.Update());

  25:                 Isolate.Verify.WasCalledWithExactArguments(() => fakeItem["Approved"] = "True");

  26:                 return true; // we called all the updates expected

  27:             }

  28:             catch (TypeMock.VerifyException)

  29:             {

  30:                 // it did not call something expected, check if it was just the not approved path

  31:                 Isolate.Verify.WasNotCalled(() => fakeItem.Update());

  32:                 return false;

  33:             }

  34:  

  35:         }

  36:     }

  37: }

I am currently working on a sample of this testing technique, that does a bit more than a simple branch on if test, I will post a set of sample code when I am done.

Testing SharePoint Workflows using TypeMock Isolator (Part 2)

Updated 12 June 2009 – I have been having problems using this technique of Typemock with Sharepoint Workflows, the workflows keep unexpectedly idling as opposed to activating. If you suffer similar problem please check for later posts as to any solutions I find. 

After reading Gil’s blog on writing simpler tests I have done some tidying of the code from my previous post. In this version I have extracted the boiler plate code to run the workflow to a static helper method and modified my tests to incorporate Gil’s comments, they are certainly more readable.

   1: [TestMethod]

   2:       public void WorkFlowSwitchOnTitle_TitleStartsWithA_SetApprovelFieldAndUpdate()

   3:       {

   4:           // Arrange

   5:           var fakeProperties = Isolate.Fake.Instance<SPWorkflowActivationProperties>();

   6:           var fakeItem = fakeProperties.Item;

   7:           Isolate.WhenCalled(() => fakeItem.Title).WillReturn("ABC");

   8:  

   9:           // we actually don't need to create this field MOSS lets us attempt to write to

  10:           // it even if not declared, it would only need to be created if we check the value in the workflow

  11:           /*

  12:           var fakeField = fakeItem.Fields["Approved"];

  13:           fakeField.DefaultValue = false.ToString();

  14:           */

  15:  

  16:           // Act

  17:           WorkflowRunner(typeof(SharePointWorkflow.Workflow1),fakeProperties);

  18:           

  19:           // Assert

  20:           Isolate.Verify.WasCalledWithExactArguments(() => fakeItem.Update());

  21:           Isolate.Verify.WasCalledWithExactArguments(() => fakeItem["Approved"] = "True");

  22:  

  23:       }

  24:  

  25:       [TestMethod]

  26:       public void WorkFlowSwitchOnTitle_TitleStartsWithZ_DoNothing()

  27:       {

  28:           // Arrange

  29:           var fakeProperties = Isolate.Fake.Instance<SPWorkflowActivationProperties>();

  30:           var fakeItem = fakeProperties.Item;

  31:           Isolate.WhenCalled(() => fakeItem.Title).WillReturn("XYZ");

  32:  

  33:           // Act

  34:           WorkflowRunner(typeof(SharePointWorkflow.Workflow1),fakeProperties);

  35:  

  36:           // Assert

  37:           Isolate.Verify.WasNotCalled(() => fakeItem.Update());

  38:  

  39:       }

  40:  

  41:  

  42:  

  43:       /// <summary>

  44:       /// A helper method to run a workflow for a test

  45:       /// </summary>

  46:       /// <param name="wfType">The type of workflow to create</param>

  47:       /// <param name="fakeProperties">The fake properties used to create the workflow</param>

  48:       private static void WorkflowRunner(Type wfType, SPWorkflowActivationProperties fakeProperties)

  49:       {

  50:           using (WorkflowRuntime workflowRuntime = new WorkflowRuntime())

  51:           {

  52:               AutoResetEvent waitHandle = new AutoResetEvent(false);

  53:               workflowRuntime.WorkflowCompleted += delegate(object sender, WorkflowCompletedEventArgs e)

  54:               {

  55:                   // don't put asserts here as will be in the wrong thread

  56:                   waitHandle.Set();

  57:               };

  58:  

  59:               workflowRuntime.WorkflowTerminated += delegate(object sender, WorkflowTerminatedEventArgs e)

  60:               {

  61:                   // don't put asserts here as will be in the wrong thread

  62:                   waitHandle.Set();

  63:               };

  64:  

  65:               // when this is called the constructor is called twice

  66:               // the first time is for validation for the workflow in this appdomain see http://odetocode.com/Blogs/scott/archive/2006/03/30/3192.aspx

  67:               // then the real construction is run, the problem is this double run means that the Isolate.Swap.NextInstance

  68:               // fails as it attaches to the first validation create, not the second real one

  69:               WorkflowInstance instance = workflowRuntime.CreateWorkflow(wfType);

  70:  

  71:               // SO for this reason we only get the swap after the first create has beend one

  72:               Isolate.Swap.NextInstance<SPWorkflowActivationProperties>().With(fakeProperties);

  73:  

  74:               // we then recreate the workflow again, this time it has already been validated so

  75:               // so the swap works

  76:               instance = workflowRuntime.CreateWorkflow(wfType);

  77:  

  78:               instance.Start();

  79:  

  80:               waitHandle.WaitOne();

  81:  

  82:               // the workflow is finished assert could go here, but will be done

  83:               // in the calling method

  84:  

  85:           }

  86:       

  87:       }

The workflow remains the same as in the previous post.

Testing SharePoint Workflows using TypeMock Isolator

Updated 12 June 2009 – I have been having problems using this technique of Typemock with Sharepoint Workflows, the workflows keep unexpectedly idling as opposed to activating. If you suffer similar problem please check for later posts as to any solutions I find. 

Updated 6 April 2009 – Also see Testing SharePoint Workflows using TypeMock Isolator (Part 2)

I have for a while been trying to test SharePoint workflows using TypeMock Isolator to mock out the SharePoint fixtures, I want to remove the dependency of having SharePoint on any test boxes where possible. I have at last got this working after getting a new version of TypeMock Isolator 5.3.0 + a fix from the very helpful team at TypeMock

My idea was to be able to build a workflow that could changed list item properties for a document e.g. the workflow could set a field called approved to true if certain criteria were met. Now as a MOSS2007 workflow is based on .NET WF I knew I could try to build upon the work I document in my previous post on TDD for WF.

My test system was as follows:

  1. I created a new SharePoint workflow, all it contained was a decision box that went down the true path if the document associated with the workflow has a title starting with the letter A
  2. In a coded action for the true path, I then set an approved property to true.

All very simple, but good enough for this test, the key methods are shown below

   1: private void IfTest(object sender, ConditionalEventArgs e)

   2:    {

   3:       var currentItem = workflowProperties.Item;        e.Result = currentItem.Title.StartsWith("A");

   4:    }

   5:  

   6:    private void TrueTask(object sender, EventArgs e)

   7:    {

   8:       var currentItem = workflowProperties.Item;

   9:       currentItem["Approved"] = true.ToString();

  10:       currentItem.Update();

  11:    }

I then created a test using the same form I did for WF based testing, I think the comments cover the key points

   1: [TestMethod]

   2:       public void WorkFlowSwitchOnTitle_TitleStartsWithA_SetApprovelField()

   3:       {

   4:           using (WorkflowRuntime workflowRuntime = new WorkflowRuntime())

   5:           {

   6:  

   7:               // Create our fake workflow and items

   8:               var fakeProperties = Isolate.Fake.Instance<SPWorkflowActivationProperties>(Members.ReturnRecursiveFakes);

   9:               var fakeItem = Isolate.Fake.Instance<SPListItem>(Members.ReturnRecursiveFakes);

  10:  

  11:               var fakeField = Isolate.Fake.Instance<SPField>(Members.ReturnRecursiveFakes);

  12:               fakeField.DefaultValue = false.ToString();

  13:               Isolate.WhenCalled(() => fakeProperties.Item).WillReturn(fakeItem);

  14:               // setup the if test

  15:               Isolate.WhenCalled(() => fakeItem.Title).WillReturn("ABC");

  16:               Isolate.WhenCalled(() => fakeItem["Approved"]).WillReturn(fakeField);

  17:  

  18:               // setup the workflow handling                AutoResetEvent waitHandle = new AutoResetEvent(false);

  19:               workflowRuntime.WorkflowCompleted += delegate(object sender, WorkflowCompletedEventArgs e)

  20:               {

  21:                   // don't put asserts here as will be in the wrong thread

  22:                   waitHandle.Set();

  23:               };

  24:  

  25:               workflowRuntime.WorkflowTerminated += delegate(object sender, WorkflowTerminatedEventArgs e)

  26:               {

  27:                   // don't put asserts here as will be in the wrong thread

  28:                   waitHandle.Set();

  29:               };

  30:  

  31:               // when this is called the constructor is called twice

  32:               // the first time is for validation for the workflow in this appdomain see http://odetocode.com/Blogs/scott/archive/2006/03/30/3192.aspx

  33:               // then the real construction is run, the problem is this double run means that the Isolate.Swap.NextInstance

  34:               // fails as it attaches to the first validation create, not the second real one

  35:               WorkflowInstance instance = workflowRuntime.CreateWorkflow(typeof(SharePointWorkflow.Workflow1));

  36:  

  37:               // SO for this reason we only get the swap after the first create has been done

  38:               Isolate.Swap.NextInstance<SPWorkflowActivationProperties>().With(fakeProperties);

  39:  

  40:               // we then recreate the workflow again, this time it has already been validated so

  41:               // the swap works

  42:               instance = workflowRuntime.CreateWorkflow(typeof(SharePointWorkflow.Workflow1));

  43:  

  44:               instance.Start();

  45:  

  46:               waitHandle.WaitOne();

  47:               // wait for the workflow to complete and then check the method expected were called                Isolate.Verify.WasCalledWithExactArguments(() => fakeItem.Update());

  48:               Isolate.Verify.WasCalledWithExactArguments(() => fakeItem["Approved"] = "True");

  49:           }

  50:  

  51:       }

 

If you try this without the fix TypeMock provided me with, the two verifies will fail, I am told this is due to a threading issue. Interesting that this is SharePoint specific as the same basic method works OK for standard WF workflows.

I also understand this fix will be in the next TypeMock release, I will update this post when I know for sure

Which way to fake an item in Typemock

I raised a question on the Typemock forum concerning a problem I was having mocking Sharepoint SPFarm objects. It was all down to which way to fake items using the various techniques in Isolator. It was interesting enough, I thought, to repeat here as a blog post.

I had written some tests for a method that got a list of SiteCollections that a user had rights to access. The key point being the need to access the static property SPFarm.Local to get a list of Sharepoint services to iterate across. If I ran each test by itself it worked; but if run as a batch in TestDriven.Net or MSTest the first passed and the rest failed.

The problem was down to how I was creating the fake SPfarm, I was using:

SPFarm fakeFarm = Isolate.Fake.Instance<SPFarm>(Members.ReturnRecursiveFakes);
Isolate.Swap.NextInstance<SPFarm>().With(fakeFarm);

when I should have used

SPFarm fakeFarm = Isolate.Fake.Instance<SPFarm>(Members.ReturnRecursiveFakes);
Isolate.WhenCalled(() => SPFarm.Local).WillReturn(fakeFarm);

Ok, I used the wrong call, but wait a minute, each of my tests were marked with the [Isolated] attribute. My understanding was this meant the Typemock system was reset between each test, and as I only call SPFarm.Local once per test were these two forms not equivalent?

This is the answer from Doron at Typemock, hope it clear up any confusion of the type I was suffering from…..

You are right in that the Isolate attribute resets fake behavior between each and every test. However, SwapNextInstance<T> is triggered if and only if a constructor for T has been called. It is not equivalent to WhenCalled() but rather complementing.
Generally speaking, you set behaviors on your fake object (or static methods) using WhenCalled() and then you can choose how to inject that fake behaviour to the code under test:
– If the code under test receives a reference to the fake behavior, you just pass it in.
– If the code under tests receives it from a third party, you fake that third party to return the fake object you set up.
– If the code under test uses ‘new’ to instantiate the dependent behavior, use SwapNextInstance to replace the next ‘new’ with the faked object

Running TypeMock based test in Team Build

If you have TypeMock Isolator based MSTests in a solution you will want them to be run as part of any CI build process.

To get this to work with Team Build you have to make sure Isolator is started in the build box at the right time (something that is done automagically behind the scenes by Visual Studio during developer testing). This is not actually that difficult as TypeMock provide some tasks for just this purpose.

Firstly you have to install Isolator on the build box (and of course license it). Then edit your tfsbuild.proj build script to include the overrides for the beforetest and aftertest targets

<!-- Import the Typemock list of tasks -->  <PropertyGroup>     <TypeMockLocation>C:\Program Files\Typemock\Isolator\5.1</TypeMockLocation> </PropertyGroup> <Import Project ="$(TypeMockLocation)\TypeMock.MSBuild.Tasks"/>  <!-- Before the tests are run start TypeMock --> <Target Name="BeforeTest">     <TypeMockStart/> </Target>  <!-- And stop it when the are finished --> <Target Name="AfterTest">     <TypeMockStop/> </Target>

Once this is done your test should run OK

Interesting news on test SharePoint

Typemock announced today a new product Isolator for Sharepoint – which allows unit testing of Sharepoint code without needing Sharepoint installed. Now this is something I have been using the full version of Isolator for of late, and there are more blog posts on the way from me, so watch this space.

So if you are a Sharepoint developer this is an important product you should a least have a look at.

It is also interesting to see the promotion mechanism being used to get the word out onto the blogs, free license to the first 50 bloggers. So I would like to state my position here, I already have a Typemock Isolator license so this post has been written without the carrot of a free license, just written by an very impressed user.

Entering a license key for Typemock Isolator if you are not administrator

To license an installation of Typemock Isolator you run the Configuration tools and type in the key, you don’t get the option to enter the key during the installation. When I tried to run this tool today I got the error

image

Now at first I thought it might be that I was on a64bit OS and it was looking in a portion of the registry for 32bit applications. However I was wrong it was far simpler than that.

I am no longer running as administrator on my development box, so when I installed Typemock I was asked for elevated privileges via the UAC and all was OK. The configuration tool also needed to run at this privileges so it could update the registry, so the simple fix is that you just need to call the tool with a RunAs option and all is OK

TypeMock Isolator, SPTypeMock and SharePoint testing

I had to work unexpectedly from home yesterday, this has given me a chance to look at TypeMock Isolator and SPTypeMock to aid in the testing of SharePoint without the normal disturbances of being in the office.

First thing I have to say is TypeMock is an amazing tool, OK it costs some money, unlike RhinoMocks, but it’s ability to mock out sealed classes that have no public constructors is essential when testing SharePoint (which seems to contain nothing but sealed classes with no public constructors).

I decided to try to retro-fit some tests to an old existing WebPart.This was a simple contact form that populated some combo-boxes from SPList collections then saved it’s results to another SPList. This had all been coded making direct calls to the SPList objects as required from within the WebPart. All very old school VB6 style, no MVC pattern, so a block of legacy code that is hard to test. However, the beauty of using this mocking framework is that all you production code remains unaltered (though as we will see there is a good argument for designing/refactoring to aid testing).

I first started with the SPTypeMock library. This CodePlex project has been produced by a pair of MOSS MVPs Carlos Segura (http://www.ideseg.com) and Gustavo Velez (http://www.gavd.net ). It provides a set of wrapper classes in the form MockSPList etc. classes that you can use to construct the the SharePoint mocks, basically they hide some of the TypeMock constructions. This means that test logic ends as follows (using the sample from CodePlex)

Mocking a List Collection
— Method to be tested:

        public static string TestMock_02()         {             string strReturn = String.Empty;             try             {                 using (SPSite mySite = new SPSite("http://MiServidor"))                 {                     using (SPWeb myWeb = mySite.OpenWeb())                     {                         int intTeller = 0;                         foreach (SPList oneList in myWeb.Lists)                         {                             Debug.WriteLine(oneList.Title);                             intTeller++;                         }                         strReturn = intTeller.ToString();                     }                 }             }             catch (Exception ex)             {                 strReturn = ex.ToString();             }             return strReturn;         }

— Mocking method:

        [TestMethod]         public void TestMethod2()         {             MockSPSite mockSite = new MockSPSite("TestSite");                       MockSPWeb mockWeb = new MockSPWeb("TestWeb");                 MockSPList mockList0 = new MockSPList("MyList0");                      MockSPList mockList1 = new MockSPList("MyList1");                   MockSPList mockList2 = new MockSPList("MyList2");             mockWeb.Lists = new MockSPListCollection(new[]                             {                    mockList0,                    mockList1,                    mockList2                });               mockSite.Mock.ExpectGetAlways("RootWeb", mockWeb.GetInstance());                SPWeb WebMocked = mockWeb.GetInstance();                   using (RecordExpectations recorder = RecorderManager.StartRecording())                 {                 SPSite SiteMocked = new SPSite("");                     recorder.ExpectAndReturn(SiteMocked.OpenWeb(), WebMocked);                 }               string expected = "3";                string actual;             actual = Program.TestMock_02();             Assert.AreEqual(expected, actual);         }

This works well, they have done a good job. You get a more readable way to express standard TypeMock structure. Yes, the SPTypeMock library is missing some bits, but is a first release and they make point out there is work to do themselves. You can always just write the mocks yourself as in basic TypeMock tests.

However, I did not stop looking here, after doing a bit more reading I started to look at  Isolators’ new AAA library (Arrange, Act, Assert) which I think first shipped with  5.1.0. This aims to hide much of the mocking process inside Isolator, by default an ‘empty fake’ is created for everything in the object tree being mocked and you just set values for the bits you care about. This makes it very easy to create a fake of a large system such as Sharepoint by using the magic Members.ReturnRecursiveFakes option

This allowed me to greatly reduce the code required to setup by tests. I create a fake SPSite (and all the object under it) and then set the values for just the items I care about for the test.

 

SPSite fakeSite = Isolate.Fake.Instance<SPSite>(Members.ReturnRecursiveFakes);
           Isolate.Swap.NextInstance<SPSite>().With(fakeSite);

           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items).WillReturnCollectionValuesOf(
               new List<SPItem> {
                   Isolate.Fake.Instance<SPItem>(),
                   Isolate.Fake.Instance<SPItem>(),
                   Isolate.Fake.Instance<SPItem>() });

           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[0]["Title"]).WillReturn("Title1");
           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[0]["Email Address"]).WillReturn("email1@email.com");

           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[1]["Title"]).WillReturn("Title2");
           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[1]["Email Address"]).WillReturn("email2@email.com");

           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[2]["Title"]).WillReturn("Title3");
           Isolate.WhenCalled(() => fakeSite.RootWeb.Lists["Centre Locations"].Items[2]["Email Address"]).WillReturn("email3@email.com");

This I think makes the unit testing of business logic within SharePoint viable without having to jump through too many hoops.

Given the choice between the SPTypeMock and the AAA syntax I think I would stick with the latter, but you never know in the future. I suppose it will all come down to which syntax gives the quickest (and maybe more importantly easiest to read) tests in the future.

I did say said I would come back to the application under test’s architecture. This simple WebPart, which contain a good selection of calls to SPLists and had client side validation has proved to be very hard to test, as you would expect. OK you have used TypeMock to get data into it, but how do you test it is in the right field? You can try to render the control but here are issues of WebPartManagers and script registration that frankly are not worth trying to fix. The better solution is some design for test, which I think for WebParts means MVC. In some ways this pattern would negate the need for TypeMock as you would create an IDataModel interface and have any test version you require, though of course you could mock this with TypeMock.

All this said I am hugely impressed by TypeMock Isolator, a really powerful tool for the testing of complex platforms like SharePoint