But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Cannot run CodeUI tests in Lab Management – getting a ’Build directory of the test run is not specified or does not exist’

Interesting user too stupid error today whist adding some CodeUI tests to a Lab Management deployment scenario.

I added the Test Case and associated it with Coded UI test in Visual Studio


I made sure my deployment build had the tests selected


I then ran my Lab Deployment build, but got the error

Build directory of the test run is not specified or does not exist.

This normally means the test VM cannot see the share containing the build. I checked the agent login on the test VM could view the drop location, that was OK, but when I looked for the assembly containing my coded UI tests was just not there.

Then I remembered……..

The Lab build can take loads of snapshots and do a sub-build of the actual product. This all very good for production scenarios, but when you are learning about Lab Management or debugging scripts it can be really slow. To speed up the process I had told my Deploy build to not take snapshots and the use the last compile/build drop it could find. I had just forgotten to rebuild my application on the build server after I had added the coded UI tests. So I rebuild that and tried again, but I got the same problem.

It turns out that though I was missing the assembly the error was before it was required. The true real error was not who the various agents were running as, but the account the test controller was running as. The key was to check the test run log. This can be accessed from the Test Run results (I seemed to have a blind spot looking for these result)


This showed problem, I had selected the default ‘Network Service’ account for the test controller and had not granted it rights to the drop location.


I changed the account to my tfs210lab account as used by the agents and all was OK.


Don’t hardcode that build option

I have been using the ExternalTestRunner 2010 Build activity I wrote. I realised that at least one of the parameters I need to set, the ProjectCollection used to publish the test results, was hard coded in my sample. It was set in the form


This is not that sensible, as this value is available using the build API as


It makes no sense to hard code the name of the server if the build system already knows it.

This simple change means that the build templates can be fair easier past between Team Projects Collections

"Program too big to fit in memory" when installing a TFS 2010 Test Controller

Just spent a while battling a problem whilst install the TFS 2010 Test Controller. When I launched the install setup program off the .ISO  I could select the Test Controller installer, but then a command prompt flashed up and exited with no obvious error. If I went into the TestControllers directory on the mounted .ISO and ran the setup from a command prompt I saw the error "program too big to fit in memory".

As the box I was trying to use only had 1Gb of memory (below the recommended minimum), I upped it to 2Gb and then to 4Gb but still got the same error.

Turns out the problem was a corrupt .ISO once I had downloaded it again, and dropped by target VM to 2Gb of memory all was fine.

Running MSDeploy to a remote box from inside a TFS 2010 Build (Part 2)

Another follow up post, this time to the one on MSDeploy. As I said in that post a better way to trigger the MSDeploy PowerShell script would be as part of the build workflow, as opposed to a post build action in the MSBuild phase. Doing it this way means if the build failed testing, after MSBuild complete, you can still choose not to run MSDeploy.

I have implemented this using an InvokeProcess call in my build workflow, which I have placed just before Gated checking logic at the end of the process template.


The if statement is there so I only deploy if a deploy location is set and all the tests passed

BuildDetail.TestStatus = Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Succeeded And
String.IsNullOrEmpty(DeployLocation) = False

The InvokeProcess filename property is

BuildDetail.DropLocation & "\_PublishedWebsites\" & WebSiteAssemblyName & "_Package\" & WebSiteAssemblyName & ".deploy.cmd"

Where “WebSiteAssemblyName” is a build argument the name of the Project that has been publish (I have not found a way to automatically detect it) e.g. BlackMarble.MyWebSite. This obviously as be set as an argument for the build if the deploy is to work

The arguments property is set to

"/M:http://" & DeployLocation & "/MSDEPLOYAGENTSERVICE /Y”

Again the “DeployLocation” is a build arguement that is the name of the server to deploy to e.g. MyServer

The Result property is set to an Integer build variable, so any error code can be returned in the WriteBuildError

This seems to work for me and I think it is neater than previous solution

How to edit a TFS 2010 build template when it contains custom activities.

I posted a while ago on using my Typemock TMockRunner Custom Activity for Team Build 2010. I left that post with the problem that if you wished to customise a template after you a had added the custom activity you had to use the somewhat complex branching model edit the XAML.

If you just followed the process in my post to put the build template in a new team project and tried to edit the XAML you got the following errors, an import namespace error and the associated inability to render part of the workflow


The best answer I have been able to find has been to put the custom activity into the GAC on the PC what you wish to edit the template on, just there nowhere else the method in the previous post is fine for build agents. So I strongly signed the custom activity assembly, used GACUTIL to put it in my GAC and was then able to load the template without any other alterations. I as also able to add it to my Visual Studio toolbox so that I could drop new instances of the external test runner onto the workflow.

Visual Studio 2010 Lab Management released announced

In the VSLive! keynote Microsoft made announcements about Lab Management, it will be RTM’d later this month and best of all it will be included as part of the benefits of the Visual Studio 2010 Ultimate with MSDN and Visual Studio Test Professional 2010 with MSDN SKUs. You can read more detail on Brian Keller’s blog

I think this is a great move on licensing, we had expect it to be purchasable addition to Visual Studio. With this change it now is consistent with TFS i.e. if you have the right SKU of Visual Studio and MSDN you get the feature. This greatly removes the barrier to entry for this technology.

I look forward to have a forthright discussion with our IT manager over Hyper-V cluster resources in the near future

Running MSDeploy to a remote box from inside a TFS 2010 Build

[Also see Running MSDeploy to a remote box from inside a TFS 2010 Build (Part 2)

A fellow MVP Ewald Hofman wrote a great post on getting an Web Application to deploy as part of a TFS 2010 build. I have been using this technique but found a few points that were not right in the original post, I assume these are down to RC/RTM issues and the fact I was trying to access a remote system.

This is what I had to do

  1. Follow the details referenced by Ewald in the post to create a MSDeploy package, for me the default setting were fine as all I wanted to deploy was a basic web site that had no need for any SQL
  2. Test that this package is works from your development PC to the target web server. If I viewed this publish profile in VS2010 I see the following

  3. Now all these details are stored in a <ProjectName>.Publish.xml file in the project root directory. However, this file is not source controlled. It is only for use on the development PC.
  4. You need to set the Site/Application name that will be used when the build server rebuilds this file. This is done by going into the project properties and onto the Package/Publish Web tab. At the bottom of the page set the IIS web application. If you don’t set this the package built will default to the Default Site/<Project Name> Deploy virtual directory, In my case I just wanted to point to the root of a dedicated II6 hosted web site called Test3.

  5. You now need to got back to Ewalds post. In the build definition you need to add the /p:DeployOnBuild=True MSBuild parameters to cause the package to be created

  6. As he says this causes the package to be created, but not deployed, to do this you need to add a post build event to the project (Or you could edit the Build Process Template to add a PowerShell step at the end, which might be a better option. I have found as I am using a gated Check-in I get the deployment prior to the build doing testing, so potentially I publish something that builds but the tests fail).

    Now this is where I found the most obvious difference in the post, and that is the path. My post build step had the following

    if "$(ConfigurationName)" == "Release" "$(TargetDir)_PublishedWebsites\$(TargetName)_Package\$(TargetName).deploy.cmd"  /M:http://hunter/MSDEPLOYAGENTSERVICE  /Y

    The first point is that the generated file path and file name is different to that in Ewald’s post. The second point is that he was trying to deploy locally to allow a CodeUI test to run, I wanted the build to be deployed to a simple IIS server (not a clever lab management environment) so I also need the /M: parameter.

    Also remember is that the build agent service user (in my case TFSBuild) must have admin rights on the box you are trying to deploy too, esle the process fails

Getting code coverage working on Team Build 2010

If you have VS2010 Premium or Ultimate [Professional corrected error in orginal post]  you have code coverage built into the test system. When you look at your test results there is a button to see the code coverage


You would think that there is easy way to use the code coverage in your automated build process using Team Build 2010, well it can done but you have to do a bit of work.

What’s on the build box?

Firstly if your build PC has only an operating system and the Team Build Agent (with or without the Build Controller service) then stop here. This is enough to build many things but not to get code coverage. The only way to get code coverage to work is to have VS2010 Premium or Ultimate also installed on the build box.

Now there is some confusion in blog posts over if you install the Visual Studio 2010 Test Agents do you get code coverage, the answer for our purposes is no. The agents will allow remote code coverage in a Lab Environment via a Test Controller, but they do not provide the bits needs to allow code coverage to be run locally during a build/unit test cycle.

Do I have a .TestSettings file?

Code Coverage is managed using your solution’s .TestSetting file. My project did not have one of these, so I had to ‘add new item’ it via add on a right click in the solution items.

The reason I had no .TestSettings file was because I started with an empty solution and added projects to it, if you start with a project, such as a web application, and let the solution be created for you automatically then there should be a .TestSettings file created.

In the test settings you need to look at the Data & Diagnostics tab and enable code coverage and then press the configure button, this is important.


On the configuration dialog will see a list of your projects and assemblies. In my case initially I only saw the first and the last rows in the graphic below. I selected the first row, the project containing my production code and tried a build.

THIS DID NOT WORK – I had to added the actual production assembly as opposed to the web site project (the middle row shown below). I think this was the key step to getting it going.

The error I got before I did this was Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems.  So if you see this message in the build report check what assemblies are flagged for code coverage.


Does my build definition know about the .TestSettings file?

You now need to make sure that build knows the .TestSettings file exists. Again this should be done automatically when you create a build (if the file exists), but on my build I had to add it manually as I created the file after the build.





So when all this is done you get to see a build with test results and code coverage.


Easy wasn’t it!