But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Speaking at QCon on TFS and Java Integration

Week after next I will be speaking at QCon London with Simon Thurman of Microsoft on “The Interoperable Platform”.

So what does that title mean? Well for me, for this session, it will be about how you can use the ALM features of TFS even when using Eclipse for Java development. So it will be a demo led session on the Teamprise tools for Eclipse and how they can allow you to build a unified development team that works in both .NET and Java.

Should be an interesting event, the list of speaker looks great. Shame I will only be there for a day

Logging results from InvokeProcess in a VS2010 Team Build

When you use the InvokeProcess activity, as I did in my Typemock post, you really need to setup the logging. This is because by default nothing will be logged other than the command line invoked, not usually the best option. There are a couple of gotta’s here that initially caused me a problem and I suspect could cause a new user of the 2010 build process a problem too.

The first is that you need to declare the variable names for the InvokeProcess to drop the output and errors into. This is done in the workflow designer putting the variable names in the relevant textboxes (there is no need to declare the variable names anywhere else) as shown below. Use any name you fancy, I used stdOutput and stdError.

image

You then need to add the WriteBuildMessage and WriteBuildError activities by dragging them from the toolbox into the hander areas of the InvokeProcess activity.

The second gotta is that the WriteBuildMessage takes a logging level parameter. This defaults to normal, this means the message will not be displayed in the standard build view (unless the build’s detail level is altered). To get ground this, as I would normally want to see the output of the process being invoked, I would set the Importance of the message to High. Remember you also need to set the Message parameter to the previously declared variable name, in my case stdOutput. This is done in the properties windows as shown below.

image

Note that you don’t need to set an importance on the WriteBuildError activity as this is automatically always displayed, you just need to set the Message parameter to stdError.

Once you make these changes and run the build, you see the output of the command line (green) in the build log as well as the command line (red). This should help with debugging your InvokeProcess activities in your build process.

image

MTLM becomes MTM

You may have noticed that Microsoft have had another burst of renaming. The tester’s tool in VS2010 started with the codename of Camaro during the CTP phase, this became Microsoft Test & Lab Manager (MTLM) in the Beta 1 and 2 and now in the RC it is call Microsoft Test Manager (MTM).

Other than me constantly referring to things by the wrong name, the main effect of this is to make searching on the Internet a bit awkward, you have to try all three names to get good coverage. In my small corner of the Internet, I will try to help by updating my existing MTLM tag to MTM and update the description appropriately.

So where have I been all week?

A bit a a double question here, physically I have been at the the MVP Summit in Redmond, having a great time with my fellow “Team System” MVPs and the Microsoft product group members.

4368620611_d1ce34e06a

But my blog has also been on and off all week, so I guess you could say my online presence has been away. This is because Black Marble has moved office and our blog server has had intermittent connectivity, which hopefully should be resolved soon.

Tonight is Agile Yorkshire

A late reminder but tonight is the monthly Agile Yorkshire meeting. This month is an open floor meeting with short presentations from members. Currently the planned subjects are:

  • REST and OpenRasta
  • Silverlight
  • F#
  • Thoughts on Test Driven Development practices
  • Behaviour Driven Development

Hope to see you there, usual place usual time (Victoria Hotel, 28 Great George St, Leeds. See here for directions, 7pm)

Is Pex and Moles the answer to Sharepoint testing?

I have got round to watching Peli de Halleux’s presentation on testing SharePoint with moles from the SharePoint Connections 2010 event in Amsterdam, very interesting. This brings a whole new set of tools to the testing of Sharepoint. I think it is best to view the subject of this presentation in two parts Pex and Moles, even though they are from the same stable; Moles being produced to enable Pex. But rather than me explaining how it all works just watch the video.

So to my thoughts, the easier bit to consider is Pex. If you can express your unit tests in a parameterised manner then this is a great tool for you. The example that Peli gives of an event handler that parses a string is a good one. We all have loads of places where this type of testing is needed, especially in Sharepoint. The problem here, as he points out, is that you need to use some form of mocking framework to allow the easy execution of these tests for both developers and automated build servers. I would usually use Typemock Isolator to provide this mocking, the problem is that Pex and Isolator at this time can’t run together. The Pex Explorer does not start the Typemock mocking interceptor, and thus far I can’t find a way to get round the problem.

So enters Moles, this is Microsoft Research’s subbing framework that ‘detour any .NET method to user-defined delegates, e.g., replace any call to the SharePoint Object Model by a user-defined delegate’. Now I find the Moles syntax is a bit strange. I suspect it is down to my past experience, but I still find the Typemock Isolator AAA syntax easier to read than Moles’. However, there are some nice wrapper classes provided to make it easier to use the Moles framework with Sharepoint.

So where does this leave us? At this time if you want to use Pex (and I certainly would like to) you have to use Moles (if you need stubbing). But you also have to remember that Pex & Moles are research projects. They are available for commercial evaluation, but at this time there seems to be no plans to productise it or roll it into Visual Studio, this means on effect no support. I don’t see either of these points as being a major barrier, as long as you make the choice to accept them knowingly.

However for ultimate flexibility it would be really nice to see Typemock Isolator being interoperable Pex, thus allowing me to use the new techniques of Pex against legacy tests already written using Isolator.

Errors Faking Sharepoint with Typemock due to assembly versions

I was doing some work today where I needed to fake out a SPWeb object. No problem you think, I am using Typemock Isolator so I just use the line

var fakeWeb = Isolate.Fake.Instance<SPWeb>();

But I got the error

Microsoft.SharePoint.SPWeb.SPWebConstructor(SPSite site, String url, Boolean bExactWebUrl, SPRequest request)
Microsoft.SharePoint.SPWeb..ctor(SPSite, site, String url)
eo.CreateFakeInstance[T](Members behavior, Constructor constructorFlag, Constructor baseConstructorFlag, Type baseType, Object[] ctorArgs)
eo.Instance[T](Members behavior)
(Points to the SPWeb web line as source of error)
TypeMock.MockManager.a(String A_0, String A_1, Object A_2, Object A_3, Boolean A_4, Object[] A_5)
TypeMock.InternalMockManager.getReturn(Object that, String typeName, String methodNAme, Object methodParameters, Boolean isInjected)
(Points to line 0 of my test class)

This seemed strange I was doing nothing clever, and something I have done many times before. Turns out the issue was the version of the Typemock assemblies I was referencing. I have referenced the 5.4 versions, once I repointed to the new 6.0 (or I suspect older 5.3 ones) all was OK.

At last, my creature it lives……..

I have at last worked all the way through setting up my portable end to end demo of  testing using Windows Test and Lab Manager. The last error I had to resolve was the tests not running in the lab environment (though working locally on the development PC). My the Lab Workflow build was recorded as a partial success i.e. it built, it deployed but all the tests failed.

I have not found a way to see the detail of why the tests failed in VS2010 Build Explorer. However, if you:

  1. Go into MTLM,
  2. Pick Testing Center
  3. Select the Test Tab
  4. Pick the Analyze Test Results link
  5. Pick the test run you want view
  6. The last item in the summary is the error message , as you can see in my case it was that the whole run failed not any of the individual tests themselves

image

So my error was “Build directory of the test run is not specified or does not exist”. This was caused because the Test Controller (for me running as Network Service) could not see the contents of the drop directory. The drop directory is where the test automation assemblies are published as part of the build. Once I gave Network Service read rights to access the \\TFS2010\Drops share my tests, and hence my build, ran to completion.

It has been a interesting journey to get this system up and running. MTLM when you initially look at it is very daunting, you have to get a lot of ducks in a row and there are many pitfalls on the way. If any part fails then nothing works, it feels like a bit of a house of cards. However if you work though it step by step I think you will come to see that the underlying architecture of how it hangs together is not as hard to understand as it initially seems. It is complex and has to be done right, but you can at least see why things need to be done. Much of this perceived complexity for me a developer is that I had to setup a number of ITPro products I am just not that familiar with such as SCOM and Hyper-V Manager. Maybe the answer is to make your evaluation of this product a joint Dev/ITPro project so you both learn.

I would say that getting the first build going (and hence the underlying infrastructure) seems to be the worst part. I feel that now I have a platform I understand reasonably, that producing different builds will not be too bad. I suspect the next raft of complexity will appear when I need a radically different test VM (or worse still a networks of VMs) to deploy and test against.

So my recommendation to anyone who is interest in this product is to get your hands dirty, you are not going to understand it by reading or watching videos, you need to build one. So find some hardware, lots of hardware!