But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Cannot run coded ui test on a TFS lab due to lack of rights to the drops folder

Whilst setting up a TFS 2012 Lab management environment for Coded UI testing we got the problem that none of the tests were running, in fact we could see no tests were even being passed to the agent in the lab

image

On the build report I clicked on  the ‘View Test Results’ which loaded the result in Microsoft Test Manager (MTM)

image

and viewed the test run log, and we saw

image

The issue, it claimed, was that the build controller did not have rights to access the drop folder containing the assembly with the CodedUI. tests.

Initially i thought the issue was the test controller was running a ‘local service’, so I changed it to the domain\tfsbuild account (which obviously has rights to the drops folder as it put the files there) but still got the same error. i was confused.

So I checked the event log on the build controller and found the following

image

The problem was my tfslab account, not the local service or tfsbuild one. So the message shown in the build report was just confusing, mentioning the wrong user. The lab account is the one configured in the test controller (yes you have to asked how had I missed that when I had been into the same tools to change the user the test controller ran as!)

image

As soon as I granted this tfslab user rights to the drops folder all was OK

Team Foundation Service RTMs

Today at Build 2012 it was announced that https://tfspreview.com has RTM'd as Team Foundation Service on https://tfs.visualstudio.com.

Up until now there has been no pricing information, which had been a barrier to some people I have spoken to as they did not want to started using something without knowing the future cost.

So to the really good news, as of now

  • It is free for up to 5 users
  • If you have an active MSDN subscription it is also free. So a team of any size can use it as long as they all have MSDN

The announcement said to look out for further price options next year.

Check the full details at Soma's blog

More fun with creating TFS 2012 SC-VMM environments

Whilst setting up new a new SC-VMM based lab environment I managed to find some new ways to fail above and beyond the ones I have found before.

We needed to build a new environment for testing CRM application, this needed to have its own DC, IIS server and a CRM server. The aim was to have this as a network isolated environment, but you have to build it first as the various VMs.

So we did the following

  • On the Hyper-V hosts managed by our SC-VMM server create 3 new VMs connect to our corporate LAN
  • Install the OS on the three VMs
  • Make one of the a DC for dev.local
  • Join the others to the DC’s domain dev.local (they are not joined to our corporate domain)
  • On the IIS box add the web server role
  • On the CRM box install Microsoft CRM

So we now have a three box domain that does what we want, but it is not network isolated. We could have used the features of SC-VMM to push these VMs into the library and hence import into the Lab Management library. However we choose to make sure we could connect to them first as an environment.

So first I tried to create a standard environment, not using SC-VMM. I had to create a local hosts file on the PC running MTM, but once this was done I could verify the environment, so all was OK. I did not actually create it.

Next I tried to create a SC-VMM based environment and this is where I hit a problem. I was basically trying to do something I have done before with all our pre-lab management test VMs, wrapper existing VMs in an environment. When I tried to do this the verification failed, saying I could not connect to any of the VMs. First we made sure file sharing was enable, firewalls were not blocking etc. All to no effect.

To cut a long story short I had a number of issues, mostly down to the reuse of names

  • The SC-VMM VM names for the VMs (e.g. LabDC) did not match the actual host name (DevDC). I had to rename the VM in SC-VMM to match the name of the host in the operating system (I am still unsure if this is a red herring and not really that important, but I think it is good practice anyway)
  • We had to have a hosts file on the MTM box with the fully qualified names for the three boxes (not just the server name). Not that this hosts entry (or could be DNS if you want) is only needed until the environment is built
  • 192.168.200.102 devcrm.dev.local
    192.168.200.103 deviis.dev.local
    192.168.200.104 devdc.dev.local

  • The name DevDC has been used on another VM that was being run on one of our developers Windows 8 Hyper-V setup. This was causing problems when MTM tried to resolve the machine name via SMB (Netbios, IP resolution was fine). Switching off this other VM fixed this, we only spotted it by using Wireshark on the PC running MTM (note you have to run the installer in Win7 compatibility mode to get it to work with Windows 8)
  • When entering the login details for development domain when creating the new environment in MTM  the user ID had to be entered as administator@dev.local and not dev\administrator

Once this was all done I could verify my environment and create it, the TFS agent was installed, but did not connect to the test controller. This is exactly as expect as details in my previous post.

I now have a few choices

  • If I don’t want to network isolate it I can install a Test Controller in the domain
  • I can save each of the three VMs into the SC-VMM library via MTM and create an isolated environment.

So I hope this helps you avoid some of the problems I have seen, I just wish that the MTM environment creation step gave out a better log file so i don’t have to second guess it or use wireshark.

TFS Test Agent cannot connect to Test Controller – Part 2

I posted last week on the problems I had had getting the test agents and controller in a TFS2012 Standard environment talking to each other and a workaround. Well after a good few email with various people at Microsoft and other consultants at Black Marble I have a whole range of workarounds solutions.

First a reminder of my architecture, and note that this could be part of the problem, it is all running on a single Hyper-V host. Remember this is a demo rig to show the features of Standard Environments. I think it is unlikely that this problem will be seen in a more ‘realistic’ environment i.e. running on multiple boxes

 

image

 

The problem is that when the test agent running on the Server2008 should request the test controller (running the on VSTFS server) should call it back on either it 169.254.x.x address or on abn address obtained via DHCP from the external virtual switch. However the problem is it is requesting a call back on 127.0.0.1, as can be seen in the error log

Unable to connect to the controller on 'vstfs:6901'. The agent can connect to the controller but the controller cannot connect to the agent because of following reason: No connection could be made because the target machine actively refused it 127.0.0.1:6910. Make sure that the firewall on the test agent machine is not blocking the connection.

The root cause

It turns out the root cause of this problem was I had edited the c:\windows\system32\drivers\etc\hosts file on the test server VM to add an entry to allow a URL used in CodedUI tests to be resolved to the localhost

127.0.0.1   www.mytestsite.com

Solution 1 – Edit the test agent config to bind to a specific address

The first solution is the one I outlined in my previous post, tell the test agent to bind to a specific IP address. Edit

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\QTAgentService.exe.config

and added a BindTo line with the correct address for the controller to call back to the agent

<appSettings>
     // other bits …
      <add key="BindTo" value="169.254.1.1"/>
</appSettings>

The problem with this solution you need to remember to edit a config file, all seems a bit complex!

Solution 2 – Don’t resolve the test URL to localhost

Change the hosts file entry used by the CodedUI test to resolve to the actual address of the test VM e.g.

169.254.1.1   www.mytestsite.com

Downside here is you need to know the test agents IP address, which depending on the system in use could change, and will certainly be different on each test VM in an environment. Again all seems a bit complex and prone to human error.

Solution 3 – Add an actual loopback entry to the hosts file.

The simplest workaround which Robert Hancock at Black Marble came up with was to add a second entry to the hosts file for the name loopback

127.0.0.1   localhost
127.0.0.1   www.mytestsite.com

Once this was done the test agent could connect, I did not have to edit any agent config files, or know the address the agent need to bind to. By far the best solution

 

So thanks to all who helped get to the bottom of this surprisingly complex issue.

TFS Test Agent cannot connect to Test Controller gives ‘No connection could be made because the target machine actively refused it 127.0.0.1:6910’

Updated 1st October – See the Part 2 post which provides more workaround solutions

Whilst setting up a  TFS 2012 Standard Lab Environment for an upcoming demo I hit a problem. Initially my environment had worked fine, I could deploy to my server VM in the environment without error. However, after a reboot of the TFS server (which has the build and test controllers on it) and the single server VM in the environment, the test agent on the VM could not connect to the test controller on the TFS SERVER. The VM’s event log showed

Unable to connect to the controller on 'tfsserver:6901'. The agent can connect to the controller but the controller cannot connect to the agent because of following reason: No connection could be made because the target machine actively refused it 127.0.0.1:6910. Make sure that the firewall on the test agent machine is not blocking the connection.

The key here was test controller was being told to call back to the test agent on 127.0.0.1 – which is obviously wrong being the loopback address.

So it seems the test agent was telling the test server the wrong IP address, not sure why it was resolving this address but I did find a workaround, on the test VM I edited 

‘C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\QTAgentService.exe.config’

and added the BindTo line with the correct address for the controller to call back to the agent

<appSettings>
     // other bits …
      <add key="BindTo" value="10.0.0.1"/>
</appSettings>

Once I restarted the test agent it connected to the controller and I could run my builds.

For more details on this config file see http://msdn.microsoft.com/en-us/library/ff934571.aspx

Type ‘InArgument(mtbwa:BuildSettings)’ of property ‘BuildSettings’ errors in TFS 2012 RTM builds

I posted a while ago that you saw errors when trying to edit TFS 2012RC build process templates in VS 2012RC if the Visual Studio class library project you were using to manage the process template editing was targeting .NET 4.5, it needed to be 4.0. Well with Visual Studio 2012 RTM this is no longer the case, in fact it is the other way around.

I have recently upgraded our TFS 2012 RC –> RTM and I today came to edit one of our build process templates (using the standard method to edit a process template with custom activities) and got the following error when I tried to open the XAML process template for editing

System.Xaml.XamlException: 'The type ‘InArgument(mtbwa:BuildSettings)’ of property ‘BuildSettings’ could not be resolved.' Line number '3' and line position '38'.

 

image

At first I assumed it was my custom activities, so I tried editing the DefaultTemplate.11.1.xaml in the same manner, but got the same problem.

Strangely I found that if I had no solution open in Visual Studio then I could just double click on the DefaultTemplate.11.1.xaml file in Source Control Explorer and it opened without error. However, if I had a solution open in the same instance of VS2012 that contained a class library project that linked to the same XAML file I got the error. Unloading the project within the solution allowed me to open the file via Source Control Explorer, reloading the project again stopped it loading.

So it all pointed to something about the containing class library project stopping referenced assemblies loading. On checking the project properties I saw that it was targeting .NET 4.0 (as required for the RC), as soon as I changed this to .NET 4.5 it was able to load all the required Team Foundation assemblies and I was able to edit both the default template and my custom build process template.

Getting Typemock Isolator running within a TFS 2012 build – part 2

I posted previously on getting Typemock 7.x running in a TFS 2012 RC build process . Well it seems the activities I previously published did not work on the TFS 2012 RTM build i.e if you do nothing other than upgrade your TFS server from RC to RTM a previously working build fails, no attempt was made to run any tests and I got the unhelpful error

TF900546: An unexpected error occurred while running the RunTests activity: 'Executor process exited.'.

Note: TF900546 seems to be the generic – test failed error number. If you see it you will usually have to look elsewhere for anything helpful.

So I assumed that the problem must be some difference with the TeamFoundation assemblies I was referencing between the RC and RTM versions, so I rebuilt my activities, all to no effect, I got the same error. So I did some more digging into the code. I found a number of issues, why these had not caused an issue before I don’t know:

Target property

If you do not specify a .NET version via the Target property of the TypeMockRegister activity it does not attempt to start interception. As setting this property every time you want to use the activity is a pain, I modified the activity so that if no Target property is passed then the value v4.0.30319 is used, the version of .NET 4.5 as it appears in the c:\windows\Microsoft.Net\framework folder.

Note Missing of the leading v if the Target value causes the TFS build agent to hang, I have no idea why.

Once this change was made the build ran and it tried to run all my tests, but the ones involving Typemock failed, with the message

Test method BuildProcessValidation.Tests.MSTestTypemockTests.DirtyTrickMockingWithTypemock_Email_is_sent_when_client_order_is_processed threw exception:
System.TypeInitializationException: The type initializer for 'f5' threw an exception. ---> System.TypeInitializationException: The type initializer for 'TypeMock.InterceptorsWrapper' threw an exception. ---> TypeMock.TypeMockException:
*** Typemock Isolator needs to be linked with Coverage Tool to run, to enable do one of the following:
   1. link the Coverage tool through the Typemock Isolator Configuration
   2. run tests via TMockRunner.exe -link
   3. use TypeMockStart tasks for MSBuild or NAnt with Link
For more information consult the documentation (see Code Coverage with Typemock Isolator topic)

On looking in the build box’s event log I saw the message

.NET Runtime version 4.0.30319.17929 - Loading profiler failed during CoCreateInstance.  Profiler CLSID: '{B146457E-9AED-4624-B1E5-968D274416EC}'.  HRESULT: 0x8007007e.  Process ID (decimal): 2068.  Message ID: [0x2504].

AutoDeployment

Basically the issue was the Typemock interceptor, the profiler, was not being started because Typemock was not installed on the build box. To prove this I manually installed Typemock on the build box and the error went away, all my tests ran. So happy my activity basically worked, I removed Typemock from the build box and the problem returned, so I know I had an autodeployment issue.

On checking the activity code again I found I was not handling the nullable boolean correctly for the AutoDeploy build argument of the type TypemockSettings. As soon as this was fixed and deployed by build leapt into life.

In summary

So I am please to say I have a working activity again, as I said in my previous post I see this as stopgap measure until Typemock Release their official version. This set of activities have had minimal testing and I am not sure the undeploy logic is working fully, but as I don’t need this feature I am not worrying about it for now.

Hope you find it useful in its current state.

Experiences upgrading our TFS2012RC to RTM

We have just completed the upgraded of our TFS2012 server from RC to RTM. All went smoothly, just a few comments worth mentioning

  • The install of TFS2012 (after the removal of the RC) required three reboots, 2 for C++ components and one for .NET 4.5. So  if seeing reboots don’t worry too much.
  • When running the upgrade wizard we got a verify warning over port 443 already being used (we had manually configured via IIS manager for our server to use 8080 and 443). We ignored this warning. However after the upgrade wizard had completed, with no errors, we found that the new web server could not start. Turns out it it had been left bound as HTTP to Port 443, so it was very confused. We just deleted this binding and re-added HTTP on 8080 and HTTPS on 433 with our wildcard certificate and it was fine. So in hindsight we should have headed the warning and removed our custom bindings.

So now off to the long job of upgrading build box, test controller and the rest.

Visual Studio 2012 RTM and the ALM Rangers SIM Ship best practice guidance

Overnight Visual Studio and TFS 2012 became available on MSDN for download; I really pleased to say that the ALM Rangers also SIM shipped all our 2012 guidance at the same time

I have been working on two of the Ranger project for the past year and have to say I have learnt a lot working with such a great crowd of Microsoft and external TFS experts. Some stuff I learnt was about the 2012 release it is true, but there was plenty on working with a a globally distributed team with a whole host of logistical and time constraints.

It has been a blast, but there is still more to do. Keep an eye open for supplementary ranger guidance releases in the next few months, but for now why not download and read the released guidance and do the hands on labs. I really do think they will answer many questions you have about TFS.