But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Cannot build a SSRS project in TFS build due to expired license

If you want to get your TFS build process to product SSRS RDL files you need to call the vsDevEnv custom activity to run Visual Studio (just like for SSIS packages). On our new TFS2013.3 based build agents this step started to fail, turns out the issue was not incorrect versions of DLLs or a some badly applied update, but that the license for Visual Studio on the build agent had expire.

I found it by looking at diagnostic logs in the TFS build web UI.

image

To be able to build BI project with Visual Studio you do need a licensed copy of Visual Studio on the build agent. You can use a trial license, but it will expire. Also remember if you license VS by logging in with your MSDN Live ID that too needs to be refreshed from time to time (that is what go me), so better to use a product key.

Version stamping Windows 8 Store App manifests in TFS build

We have for a long time used the TFSVersion custom build activity to stamp all our TFS builds with a unique version number that matches out build number. However, this only edits the AssemblyInfo.cs file. As we are now building more and more Windows 8 Store Apps we also need to edit the XML in the Package.appxmanifest files used to build the packages too. Just like a Wix MSI project it is a good idea the package version matches some aspect of the assemblies it contains. We need to automate the update of this manifest as people too often forget to increment the version, causing confusion all down the line.

Now I could have written a new TFS custom activity to do the job, or edited the existing one, but both options seemed a poor choice. We all know that custom activity writing is awkward and a pain to support going forward. So I decided to use the hooks in the 2013 generation build process template to just call a custom PowerShell script to do the job.

I added a PreBuildScript.PS1 file as a solution item to my solution.

I placed the following code in the file. It uses the TFS environment variables to get the build location and version; using these to find and edit the manifest files. The only gotcha is files on the build box are read only (it is a server workspace) so the manifest file has to be set it to allow it to be written back too.

# get the build number, we assume the format is Myproject.Main.CI_1.0.0.18290
# where the version is set using the TFSVersion custom build activity (see other posts)

$buildnum = $env:TF_BUILD_BUILDNUMBER.Split('_')[1]
# get the manifest file paths
$files = Get-ChildItem -Path $env:TF_BUILD_BUILDDIRECTORY -Filter "Package.appxmanifest" -Recurse
foreach ($filepath in $files)
{
    Write-Host "Updating the Store App Package '$filepath' to version ' $buildnum '"
   # update the identity value
  
$XMLfile=NEW-OBJECT XML
    $XMLfile.Load($filepath.Fullname)
    $XMLFile.Package.Identity.Version=$buildnum
   # set the file as read write
    Set-ItemProperty $filepath.Fullname -name IsReadOnly -value $false
    $XMLFile.save($filepath.Fullname)
}

Note that any output sent via Write-Host will only appear in the diagnostic log of TFS. If you use Write-Error (or errors are thrown) these messages will appear in the build summary, but the build will not fail, but will be marked as a partial success.

Once this file was checked in i was able to reference the file in the build template

image

The build could not be run and got my Windows 8 Store packages with the required version number

Publishing more than one Azure Cloud Service as part of a TFS build

Using the process in my previous post you can get a TFS build to create the .CSCFG and .CSPKG files needed to publish a Cloud Service. However, you hit a problem if your solution contains more that one Cloud Service project; as opposed to a single cloud service project with multiple roles, which is not a problem.

The method outlined in the previous post drops the two files into a Packages folder under the drops location. The .CSPKG files are fine, as they have unique names. However there is only one ServiceConfiguration.cscfg, whichever one was created last.

Looking in the cloud service projects I could find no way to rename the ServiceConfiguration file. It looks like it is like a app.config or web.config file i.e. it’s name is hard coded.

The only solution I could find was to add a custom target that is set to run after the publish target. This was added to the end of each .CCPROJ files using a text editor just before the closing </project>

 <Target Name="CustomPostPublishActions" AfterTargets="Publish">
    <Exec Command="IF '$(BuildingInsideVisualStudio)'=='true' exit 0
    echo Post-PUBLISH event: Active configuration is: $(ConfigurationName) renaming the .cscfg file to avoid name clashes
    echo Renaming the .CSCFG file to match the project name $(ProjectName).cscfg
    ren $(OutDir)Packages\ServiceConfiguration.*.cscfg $(ProjectName).cscfg
    " />
  </Target>
   <PropertyGroup>
    <PostBuildEvent>echo NOTE: This project has a post publish event</PostBuildEvent>
  </PropertyGroup>

 

Using this I now get unique name for the .CSCFG files as well as for .CSPKG files in my drops location. All ready for Release Management to pickup

Notes:

  • I echo out a message in the post build event too just as a reminder that I have added a custom target that cannot be seen in Visual Studio, so is hard to discover
  • I use an if test to make sure the commands are only run on the TFS build box, not on a local build. The main reason for this is the path names are different for local builds as opposed to TFS build. If you do want a rename on a local build you need to change the $(OutDir)Packages path to $(OutDir)app.publish. However, it seemed more sensible to leave the default behaviour occur when running locally

Getting the correct path and name for a project to pass as an MSBuild argument in TFS Build

I have been sorting out some builds for use with Release Management that include Azure Cloud Solutions. To get the correct packages built by TFS I have followed the process in my past blog post. The problem was I kept getting the build error

The target "Azure Packages\BlackMarble.Win8AppBuilder.AzureApi" does not exist in the project.

The issue was I could not get the solution folder/project name right for the MSBUILD target parameter. Was it the spaces in the folder? I just did not know.

The solution was to check the .PROJ file that was actually being run by MSBUILD. As you may know a .SLN file is not in MSBUILD format so you can’t just open it in notepad and look (unlike a .CSPROJ or .VBPROJ files), it is created by MSBUILD on the fly. To see this generated code, at a developer’s command prompt, run the following commands

cd c:\mysolutionroot
Set MSBuildEmitSolution=1
msbuild

When the MSBUILD command is run, whether the build works or not, there should be mysolution.sln.metaproj  file created. If you look in this file you will see the actual targets MSBUILD thinks it is dealing with.

In my case I could see

<Target Name="Azure Packages\BlackMarble_Win8AppBuilder_AzureApi:Publish">

So the first issue was my . were replaced by _

I changed my MSBUILD target argument to that shown in the file, but still had a problem. However, once I changed by space in the solution folder to %20 all was OK. So my final MSBUILD argument was

/t:Azure%20Packages\BlackMarble_Win8AppBuilder_AzureApi:Publish

image

Build failing post TFS 2013.3 upgrade with ‘Stack empty. (type InvalidOperationException)’

Just started seeing build error on a build that was working until we upgraded the build agent to TFS 2013.3

Exception Message: Stack empty. (type InvalidOperationException)
Exception Stack Trace:    at Microsoft.VisualStudio.TestImpact.Analysis.LanguageSignatureParser.NotifyEndType()
   at Microsoft.VisualStudio.TestImpact.Analysis.SigParser.ParseType()
   at Microsoft.VisualStudio.TestImpact.Analysis.SigParser.ParseRetType()
   at Microsoft.VisualStudio.TestImpact.Analysis.SigParser.ParseMethod(Byte num1)
   at Microsoft.VisualStudio.TestImpact.Analysis.SigParser.Parse(Byte* blob, UInt32 len)
   at Microsoft.VisualStudio.TestImpact.Analysis.LanguageSignatureParser.ParseMethodName(MethodProps methodProps, String& typeName, String& fullName)
   at Microsoft.VisualStudio.TestImpact.Analysis.AssemblyMethodComparer.AddChangeToList(DateTime now, List`1 changes, CodeChangeReason reason, MethodInfo methodInfo, MetadataReader metadataReader, Guid assemblyIdentifier, SymbolReader symbolsReader, UInt32 sourceToken, LanguageSignatureParser& languageParser)
   at Microsoft.VisualStudio.TestImpact.Analysis.AssemblyMethodComparer.CompareAssemblies(String firstPath, String secondPath, Boolean lookupSourceFiles)
   at Microsoft.TeamFoundation.TestImpact.BuildIntegration.BuildActivities.GetImpactedTests.CompareBinary(CodeActivityContext context, String sharePath, String assembly, IList`1 codeChanges)
   at Microsoft.TeamFoundation.TestImpact.BuildIntegration.BuildActivities.GetImpactedTests.CompareBuildBinaries(CodeActivityContext context, IBuildDefinition definition, IList`1 codeChanges)
   at Microsoft.TeamFoundation.TestImpact.BuildIntegration.BuildActivities.GetImpactedTests.Execute(CodeActivityContext context)
   at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
   at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)

I assume the issue is a DLL mismatch between what is installed in as part of the build agent and something in the 2012 generation build process template in use.

As an immediate fix, until I get a chance to swap the template to a newer one, was to disable Test Impact Analysis, which I was not using for this project anyway.

image

Once I did this my build completed OK with the tests ran OK

Getting the Typemock TFS build activities to work on a TFS build agent running in interactive mode

Windows 8 store applications need to be built on a TFS build agent running in interactive mode if you wish to run any tests. So whilst rebuilding all our build systems I decided to try to have all the agents running interactive. As we tend to run one agent per VM this was not going to be a major issue I thought.

However, whilst testing we found that any of our builds that use the Typemock build activities failed when the build agent was running interactive, but work perfectly when it was running as a service. The error was

 

Exception Message: Access to the registry key 'HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\TypeMock' is denied. (type UnauthorizedAccessException)
Exception Stack Trace:    at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
   at Microsoft.Win32.RegistryKey.CreateSubKeyInternal(String subkey, RegistryKeyPermissionCheck permissionCheck, Object registrySecurityObj, RegistryOptions registryOptions)
   at Microsoft.Win32.RegistryKey.CreateSubKey(String subkey, RegistryKeyPermissionCheck permissionCheck)
   at Configuration.RegistryAccess.CreateSubKey(RegistryKey reg, String subkey)
   at TypeMock.Configuration.IsolatorRegistryManager.CreateTypemockKey()
   at TypeMock.Deploy.AutoDeployTypeMock.Deploy(String rootDirectory)
   at TypeMock.CLI.Common.TypeMockRegisterInfo.Execute()
   at TypeMock.CLI.Common.TypeMockRegisterInfo..ctor()   at System.Activities.Statements.Throw.Execute(CodeActivityContext context)
   at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
   at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)

 

So the issue was registry access. Irrespective of whether running interactive or as a service I used the same domain service account, which was a local admin on the build agent. The only thing that changed as the mode of running.

After some thought I focused on UAC being the problem, but disabling this did not seem to fix the issue. I was stuck or so I thought.

However, Robert Hancock unknown to me, was suffering a similar problem with a TFS build that included a post build event that was failing to xcopy a Biztalk custom functoid DLL to ‘Program Files’. He kept getting an ‘exit code 4 access denied’ error when the build agent was running interactive. Turns out the solution he found on Daniel Petri Blog also fixed my issues as they were both UAC/desktop interaction related.

The solution was to create a group policy for the build agent VMs that set the following

  • User Account Control: Behavior of the elevation prompt for administrators in Admin Approval Mode - Set its value to Elevate without prompting.
  • User Account Control: Detect application installations and prompt for elevation - Set its value to Disabled.
  • User Account Control: Only elevate UIAccess applications that are installed in secure locations - Set its value to Disabled.
  • User Account Control: Run all administrators in Admin Approval Mode - Set its value to Disabled.

Once this GPO was pushed out to the build agent VMs and they were rebooted my Typemock based build and Robert Biztalk builds all worked as expected

AddBizTalkHiddenReferences error in TFS build when installing ProjectBuildComponent via a command line setup

I have been trying to script the installation of all the tools and SDKs we need on our TFS Build Agent VMs. This included BizTalk. A quick check on MSDN showed the setup command line parameter I need to install the build components was

 

/ADDLOCAL ProjectBuildComponent

So I ran this via my VMs setup PowerShell script, all appeared OK, but when I tried a build I got the error

 

C:\Program Files (x86)\MSBuild\Microsoft\BizTalk\BizTalkCommon.targets (189): The "AddBizTalkHiddenReferences" task failed unexpectedly.
System.ArgumentNullException: Value cannot be null.
Parameter name: path1
   at System.IO.Path.Combine(String path1, String path2)
   at Microsoft.VisualStudio.BizTalkProject.Base.HiddenReferencesHelper.InitializeHiddenReferences()
   at Microsoft.VisualStudio.BizTalkProject.Base.HiddenReferencesHelper.get_HiddenReferences()
   at Microsoft.VisualStudio.BizTalkProject.Base.HiddenReferencesHelper.GetHiddenReferencesNotAdded(IList`1 projectReferences)
   at Microsoft.VisualStudio.BizTalkProject.BuildTasks.AddBizTalkHiddenReferences.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__20.MoveNext()

The strange thing is, if I run the BizTalk installer via the UI and select just the ‘Project Build Components’ my build did not give this error.

On checking the Biztalk setup logs I saw that the UI based install does not run

 

/ADDLOCAL ProjectBuildComponent

but

 

/ADDLOCAL WMI,BizTalk,AdditionalApps,ProjectBuildComponent

Once this change was made to my PowerShell script the TFS build worked OK

Automating TFS Build Server deployment with SCVMM and PowerShell

Rik recently posted about the work we have done to automatically provision TFS build agent VMs. This has come out of us having about 10 build agents on our TFS server all doing different jobs, with different SDKs etc. When we needed to increase capacity for a given build type we had a problems, could another agent run the build? what exactly was on the agent anyway? An audit of the boxes made for horrible reading, there were very inconsistent.

So Rik automated the provision of new VMs and I looked at providing a PowerShell script to install the base tools we needed  on our build agents, knowing this list is going to change a good deal over time. After some thought, for our first attempt we picked

  • TFS itself (to provide the 2013.2 agent)
  • Visual Studio 2013.2 – you know you always end up installing it in the end to get SSDT, SDK and MSBuild targets etc.
  • WIX 3.8
  • Azure SDK 2.3 for Visual Studio 2013.2 – Virtually all our current projects need this. This is actually why we have had capacity issue on the old build agents as this was only installed on one.

Given this basic set of tools we can build probably 70-80% of our solutions. If we use this as the base for all build boxes we can then add extra tools if required manually, but we expect we will just end up adding to the list of items installed on all our build boxes, assuming the cost of installing the extra tools/SDKs is not too high. Also we will try to auto deploy tools as part of our build templates where possible, again reducing what needs to be placed on any given build agent.

Now the script I ended up with is a bit rough and ready but it does the job. I think in the future a move to DSC might help in this process, but I did not have time to write the custom resources now. I am assuming this script is going to be a constant work in progress as it is modified for new releases and tools.  I did make the effort to make all the steps check to see if they needed to be done, thus allowing the re-running of the script to ‘repair’ the build agent. All the writing to the event log is to make life easier for Rik when working out what is going on with the script, especially useful due to the installs from ISOs being a bit slow to run.

 

# make sure we have a working event logger with a suitable source
Create-EventLogSource -logname "Setup" -source "Create-TfsBuild"
write-eventlog -logname Setup -source Create-TfsBuild -eventID 6 -entrytype Information -message "Create-Tfsbuild started"

# add build service as local admin, not essential but make life easier for some projects
Add-LocalAdmin -domain "ourdomain" -user "Tfsbuilder"





# Install TFS, by mounting the ISO over the network and running the installer

# The command ‘& $isodrive + ":\tfs_server.exe" /quiet’ is run

# In the function use a while loop to see when the tfconfig.exe file appears and assume the installer is done – dirty but works

# allow me to use write-progress to give some indication the install is done.
Write-Output "Installing TFS server"
Add-Tfs "\\store\ISO Images\Visual Studio\2013\2013.2\en_visual_studio_team_foundation_server_2013_with_update_2_x86_x64_dvd_4092433.iso"

Write-Output "Configuring TFS Build"
# clear out any old config – I found this helped avoid error when re-running script

# A System.Diagnostics.ProcessStartInfo object is used to run the tfsconfig command with the argument "setup /uninstall:All"

# ProcessStartInfo is used so we can capture the error output and log it to the event log if required
Unconfigure-Tfs


# and reconfigure, again using tfsconfig, this time with the argument "unattend /configure  /unattendfile:config.ini", where

# the config.ini has been created with tfsconfig unattend /create flag (check MSDN for the details)
Configure-Tfs "\\store\ApplicationInstallers\TFSBuild\configsbuild.ini"

# install vs2013, again by mounting the ISO running the installer, with a loop to check for a file appearing
Write-Output "Installing Visual Studio"
Add-VisualStudio "\\store\ISO Images\Visual Studio\2013\2013.2\en_visual_studio_premium_2013_with_update_2_x86_dvd_4238022.iso"

# install wix by running the exe with the –q options via ProcessStartInfo again.
Write-Output "Installing Wix"
Add-Wix "\\store\ApplicationInstallers\wix\wix38.exe"

# install azure SDK using the Web Platform Installer, checking if the Web PI is present first and installing it if needed

# The Web PI installer lets you ask to reinstall a package, if it is it just ignores the request, so you don’t need to check if Azure is already installed
Write-Output "Installing Azure SDK"
Add-WebPIPackage "VWDOrVs2013AzurePack"

write-eventlog -logname Setup -source Create-TfsBuild -eventID 7 -entrytype Information -message "Create-Tfsbuild ended"
Write-Output "End of script"

 

So for a first pass this seems to work, I now need to make sure all our build can use this cut down build agent, if they can’t do I need to modify the build template? or do I need to add more tools to our standard install? or decide if it is going to need a special agent definition?

Once this is all done the hope is that when all the TFS build agents need patching for TFS 2013.x we will just redeploy new VMs or run a modified script to silently do the update. We shall see if this delivers on that promise

MSBuild targeting a project in a solution folder

Whilst working on an automated build where I needed to target a specific project I hit a problem. I would normally expect the MSBuild argument to be

/t:MyProject:Build

Where I want to build the project Myproject in my solution and perform the Build target (which is probably the default anyway).

However, my project was in a solution folder. The documentation says the you should be able to use for form

/t:TheSolutionFolder\MyProject:Build

but I kept getting the error the project did not exist.

Once I changed to

/t:TheSolutionFolder\MyProject

it worked, the default build target was run, which was OK as this was Build the one I wanted.

Not sure why occurred, maybe I should steer clear of solution folders?

Building Azure Cloud Applications on TFS

If you are doing any work with Azure Cloud Applications there is a very good chance you will want your automated build process to produce the .CSPKG deployment file, you might even want it to do the deployment too.

On our TFS build system, it turns out this is not a straight forward as you might hope. The problem is that the MSbuild publish target that creates the files creates them in the $(build agent working folder)\source\myproject\bin\debug folder. Unlike the output of the build target which puts them in the $(build agent working folder)\binaries\ folder which gets copied to the build drops location. Hence though the files are created they are not accessible with the rest of the built items to the team.

I have battled to sort this for a while, trying to avoid the need to edit our customised TFS build process template. This is something we try to avoid where possible, favouring environment variables and MSbuild arguments where we can get away with it. There is no point denying that editing build process templates is a pain point on TFS.

The solution – editing the process template

Turns out a colleague had fixed the same problem a few projects ago and the functionality was already hidden in our standard TFS build process template. The problem was it was not documented; a lesson for all of us, that it is a very good idea to put customisation information in a searchable location so others find customisations that are not immediate obvious. Frankly this is one of the main purposes of this blog, somewhere I can find what I did that years, as I won’t remember the details.

Anyway the key is to make sure the publish target for the MSBbuild uses the correct location to create the files. This is done using a pair of MSBuild arguments in the advanced section of the build configuration

  • /t:MyCloudApp:Publish -  this tells MSbuild to perform the publish action for just the project MyCloudApp. You might be able to just go /t:Publish if only one project in your solution has a Publish target
  • /p:PublishDir=$(OutDir) - this is the magic. We pass in the temporary variable $(OutDir). At this point we don’t know the target binary location as it is build agent/instance specific, customisation in the TFS build process template converts this temporary value to the correct path.

In the build process template in the Initialize Variable sequence within Run on Agent add a If Activity.

image

  • Set the condition to MSBuildArguments.Contains(“$(OutDir)”)
  • Within the true branch add an Assignment activity for the MSBuildArguments variable to MSBuildArguments.Replace(“$(OutDir)”, String.Format(“{0}\{1}\\”, BinariesDirectory, “Packages”))

This will swap the $(OutDir) for the correct TFS binaries location within that build.

After that it all just works as expected. The CSPKG file etc. ends up in the drops location.

Other things that did not work (prior to TFS 2013)

I had also looked a running a PowerShell script at the end of the build process or adding an AfterPublish target within the MSBuild process (by added it to the project file manually) that did a file copy. Both these methods suffered the problem that when the MSBuild command ran it did not know the location to drop the files into. Hence the need for the customisation above.

Now I should point out that though we are running TFS 2013 this project was targeting the TFS 2012 build tools, so I had to use the solution outlined above, a process template edit. However, if we had been using the TFS 2013 process template as our base for customisation then we would have had another way to get around the problem.

TFS 2013 exposes the current build settings as environment variables. This would allow us to use a AfterPublish MSBuild Target something like

<Target Name="CustomPostPublishActions" AfterTargets="AfterPublish" Condition="'$(TF_BUILD_DROPLOCATION)' != ''">
  <Exec Command="echo Post-PUBLISH event: Copying published files to: $(TF_BUILD_DROPLOCATION)" />
  <Exec Command="xcopy &quot;$(ProjectDir)bin\$(ConfigurationName)\app.publish&quot; &quot;$(TF_BUILD_DROPLOCATION)\app.publish&quot; /y " />
</Target>

So maybe a simpler option for the future?

The moral of the story document your customisations and let your whole team know they exist