But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

MSDeploy Parameters.xml can only replace web.config values is a value is already set

If you are using a parameters.xml file to set value with MSDeploy I have just found a gotcha. You need some value in the web.config file, not just an empty XML tag, else the replacement fails. So to explain…

I had the following parameters.xml file, and use Release Management to replace the __TAG__ values at deployment time.

<parameters>
  <parameter name="Domain" description="Please enter the name of the domain" defaultvalue="__Domain__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Web.Properties.Settings/setting[@name='Domain']/value/text()" />
  </parameter>

  <parameter name="AdminGroups" description="Please enter the name of the admin group" defaultvalue="__AdminGroups__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Web.Properties.Settings/setting[@name='AdminGroups']/value/text()" />
  </parameter>
</parameters>

If my web.config file (in the MSDeploy package to be transformed) was set to

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>Blackmarble</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value />
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

or

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>Blackmarble</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value></value>
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

only the Domain setting was set.

To get both set I had to have a value for each property, even though they were being reset at deployment.

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>DummyDomain</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value>DummyAdmins</value>
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

Never seen that one before.

A Visual Studio Extension to create MSDeploy parameters.xml files

When you using MSdeploy you should create a parameters.xml file that exposes your web.config settings at the time of installation. This enables good deployment habits, build the product one and then set system specific values using deployment tools. The problem is that this parameters.xml file is a pain to write, it is a series of XML blocks that contain XPath to find the entries to replace, typo’s are easy to introduce.

A ripe candidate for automation, but I could not find a tool to do it, so I wrote one for Visual Studio 2013 and 2015. You can find the source on GitHub andthe actual VSIX package in the Visual Studio Gallery.

So what does it do?

Once it is installed, if you right click on a web.config file you will see a context menu option to generate a parameters.xml file, click it, if the file does not exist it will be generated and added to the current project. Entries will be made for all appSettings and any custom applicationSettings blocks found in the web.config. The actual web.config values will be replaced with __TAGS__ to be set via Release Management or your tool of choice.

So the web.config file

<configuration>
  <applicationSettings>
    <Service.Properties.Settings>
      <setting name="Directory1" serializeAs="String">
        <value>C:\ABC1111</value>
      </setting>
      <setting name="Directory2" serializeAs="String">
        <value>C:\abc2222</value>
      </setting>
    </Service.Properties.Settings>
  </applicationSettings>
  <appSettings>
    <add key="APPSETTING1" value="123" />
    <add key="AppSetting2" value="456" />
  </appSettings>
</configuration>  

it generates the parameters.xml

<parameters>
  <parameter name="APPSETTING1" description="Description for APPSETTING1" defaultvalue="__APPSETTING1__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/appSettings/add[@key='APPSETTING1']/@value" />
  </parameter>

  <parameter name="AppSetting2" description="Description for AppSetting2" defaultvalue="__APPSETTING2__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/appSettings/add[@key='AppSetting2']/@value" />
  </parameter>

  <parameter name="Directory1" description="Description for Directory1" defaultvalue="__DIRECTORY1__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Service.Properties.Settings/setting[@name='Directory1']/value/text()" />
  </parameter>

  <parameter name="Directory2" description="Description for Directory2" defaultvalue="__DIRECTORY2__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Service.Properties.Settings/setting[@name='Directory2']/value/text()" />
  </parameter>

</parameters>

If a parameters.xml file already exists then you are prompted first if you wish to replace it, if you say no, then you  are prompted if you wish to add any new entries in the web.config, or do nothing.

All the work is done via an XSL Transform, so if you need to transform extra settings just add to the embedded XSLT resource and rebuild the VSIX package.

So the tool won’t do everything, but should get you close to the file you need.

Running StyleCop from the command line and in a TFS 2015 vNext build

Virtually any automated build will require some customisation beyond a basic compile. So as part of my upcoming Techorama session on TFS 2015 vNext build I need a demo of using a custom script as part of the build process. Two common customisations we use are version stamping of assemblies and running code analysis tools. For vNext build there is already a sample of version stamping, so I thought getting StyleCop running would be a good sample.

The problem

Customisation in vNext build is based around running a script, in the case of a Windows based build agents this a PowerShell script. The problem with StyleCop is that it does not provide a command line iterface. The  StyleCop CodePlex project provides only a Visual Studio add-in. There is also the ALM Ranger’s TFS community custom build  activity, but I could find no current command line interface projects.

So I needed to build one.

Step 1 – Create a command line

So my first step was to create a command line version of StyleCop. I chose to use the community build activity as a starting point. I had planned to do this all in PowerShell, but quickly found that the conversion of parameter object types and the handling of the events StyleCop uses was a bit messy. So I decided to write a wrapper class in C# that presented the same parameters as the old TFS build activity, basically take the old code and remove the Windows Workflow logic. I then provided a Main (args) method to expose the object to the command line such that it was easy to provide the required parameters.

This can all be found on my GitHub site.

Note on solution structure: As I wanted this to work for PowerShell and the command prompt I had to place the Main(args[]) method .EXE entry point in a project that built an EXE and all the rest of the wrapper code in one that built a .DLL. This is because you cannot load a type in PowerShell using add-type from an assembly built as an EXE, you get a EXTENSION_NOT_SUPPORTED exception. It means there are two projects (a DLL and an EXE) when I would really have like a single one (the EXE)

So I now had a command line I could call from my PowerShell script

StyleCopCmdLine --f="File1.cs" "File2.cs" --s="AllSettingsEnabled.StyleCop"

A good starting point. However,  more a TFS build it makes more sense to call StyleCop directly in the PowerShell, why shell out to a command prompt to run an EXE when your can run the code directly in PowerShell?

Step 2 – Create a simple PowerShell script

The PowerShell required to run StyleCop using the wrapper is simple, just providing the same parameters as used for the EXE.

Add-Type -Path "StyleCopWrapper.dll"

 

$scanner = new-object StyleCopWrapper.Wrapper
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.XmlOutputFile = "$pwd\out.xml"
$scanner.LogFile = "$pwd\log.txt"
$scanner.SourceFiles =  @("file1.cs", "file2.cs") )
$scanner.SettingsFile = "settings.stylecop"
$scanner.AdditionalAddInPaths = @("C:\Program Files (x86)\StyleCop 4.7" )
$scanner.TreatViolationsErrorsAsWarnings = $false

$scanner.Scan()

write-host ("Succeeded [{0}]" -f $scanner.Succeeded)
write-host ("Violation count [{0}]" -f $scanner.ViolationCount)

See the GitHub site’s WIKI for the usage details.

Step 3 – Create a vNext build PowerShell script

So now we have the basic tools we need to run StyleCop from a TFS vNext build, but we do need a more complex script.

The script you use is up to you, mine looks for .csproj files and runs StyleCop recursively from the directories containing the .csproj files. This means I can have a different  setting.stylecop file for each project. In general I have more strict rules on production code than unit test e.g. for unit tests I am not bother about the XML method documentation, but for production code I make sure they are present and match the method parameters.

Note: As the script just uses parameters and environment variable it is easy to test outside TFS build, a great improvement over the old build system

#
# Script to allow StyleCop to be run as part of the TFS vNext build
#
[CmdletBinding()]
param
(
    # We have to pass this boolean flag as string, we cast it before we use it
    # have to use 0 or 1, true or false
    [string]$TreatStyleCopViolationsErrorsAsWarnings = 'False'
)

# local test values, should be commented out in production
#$Env:BUILD_STAGINGDIRECTORY = "C:\drops"
#$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"

if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_STAGINGDIRECTORY))
{
    Write-Error "You must set the following environment variables"
    Write-Error "to test this script interactively."
    Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY = "C:\drops"'
    exit 1
}

# pickup the build locations from the environment
$stagingfolder = $Env:BUILD_STAGINGDIRECTORY
$sourcefolder = $Env:BUILD_SOURCESDIRECTORY

# have to convert the string flag to a boolean
$treatViolationsErrorsAsWarnings = [System.Convert]::ToBoolean($TreatStyleCopViolationsErrorsAsWarnings)

Write-Host ("Source folder (`$Env)  [{0}]" -f $sourcefolder) -ForegroundColor Green
Write-Host ("Staging folder (`$Env) [{0}]" -f $stagingfolder) -ForegroundColor Green
Write-Host ("Treat violations as warnings (Param) [{0}]" -f $treatViolationsErrorsAsWarnings) -ForegroundColor Green
 
# the overall results across all sub scans
$overallSuccess = $true
$projectsScanned = 0
$totalViolations = 0


# load the StyleCop classes, this assumes that the StyleCop.DLL, StyleCop.Csharp.DLL,
# StyleCop.Csharp.rules.DLL in the same folder as the StyleCopWrapper.dll
Add-Type -Path "StyleCop\StyleCopWrapper.dll"
$scanner = new-object StyleCopWrapper.Wrapper

# Set the common scan options,
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.AdditionalAddInPaths = @($pwd) # in in local path as we place stylecop.csharp.rules.dll here
$scanner.TreatViolationsErrorsAsWarnings = $treatViolationsErrorsAsWarnings

# look for .csproj files
foreach ($projfile in Get-ChildItem $sourcefolder -Filter *.csproj -Recurse)
{
   write-host ("Processing the folder [{0}]" -f $projfile.Directory)

   # find a set of rules closest to the .csproj file
   $settings = Join-Path -path $projfile.Directory -childpath "settings.stylecop"
   if (Test-Path $settings)
   {
        write-host "Using found settings.stylecop file same folder as .csproj file"
        $scanner.SettingsFile = $settings
   }  else
   {
       $settings = Join-Path -path $sourcefolder -childpath "settings.stylecop"
       if (Test-Path $settings)
       {
            write-host "Using settings.stylecop file in solution folder"
            $scanner.SettingsFile = $settings
       } else
       {
            write-host "Cannot find a local settings.stylecop file, using default rules"
            $scanner.SettingsFile = "." # we have to pass something as this is a required param
       }
   }

   $scanner.SourceFiles =  @($projfile.Directory)
   $scanner.XmlOutputFile = (join-path $stagingfolder $projfile.BaseName) +".stylecop.xml"
   $scanner.LogFile =  (join-path $stagingfolder $projfile.BaseName) +".stylecop.log"
   
   # Do the scan
   $scanner.Scan()

    # Display the results
    Write-Host ("`n")
    write-host ("Base folder`t[{0}]" -f $projfile.Directory) -ForegroundColor Green
    write-host ("Settings `t[{0}]" -f $scanner.SettingsFile) -ForegroundColor Green
    write-host ("Succeeded `t[{0}]" -f $scanner.Succeeded) -ForegroundColor Green
    write-host ("Violations `t[{0}]" -f $scanner.ViolationCount) -ForegroundColor Green
    Write-Host ("Log file `t[{0}]" -f $scanner.LogFile) -ForegroundColor Green
    Write-Host ("XML results`t[{0}]" -f $scanner.XmlOutputFile) -ForegroundColor Green

    $totalViolations += $scanner.ViolationCount
    $projectsScanned ++
   
    if ($scanner.Succeeded -eq $false)
    {
      # any failure fails the whole run
      $overallSuccess = $false
    }

}

# the output summary
Write-Host ("`n")
if ($overallSuccess -eq $false)
{
   Write-Error ("StyleCop found [{0}] violations across [{1}] projects" -f $totalViolations, $projectsScanned)
}
elseif ($totalViolations -gt 0 -and $treatViolationsErrorsAsWarnings -eq $true)
{
    Write-Warning ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned)
}
else
{
   Write-Host ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned) -ForegroundColor Green
}

Step 4 – Adding a the script to the repo

To use the script it needs (and any associated files) to be placed in your source control. In my case it meant I create a folder called StyleCop off the root of my TFS 2015 CTP’s Git repo and in it placed the following files

  • PowerShell.ps1 – my script file
  • StyleCop.dll – the main StyleCop assembly taken from c:\program files (x86)\StyleCop 4.7. By placing it here it means we don’t need to actually install StyleCop on the build machine
  • StyleCop.csharp.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCop.csharp.rules.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCopWrapper.dll – the wrapper assembly from my GitHub site

Step 5 – Adding the script to a build process

Once the script is in the repo adding a new step to a vNext build is easy.

  • In a browser select the Build.vNext menu options
  • The build explorer will be show, right click on the build you wish to add a step to and select edit
  • Press the ‘Add build step’ button. The list of steps will be show, pick PowerShell

    image
  • As the script is in the repo we can reference it in the new step. in my case I set the script file name to

                          StyleCop/PowerShell.ps1
  • My script takes one parameter, if we should treat StleCop violations as warnings, this is set as the script argument. Note I am using a build variable $(ViolationsAsWarnings) set to a string value ‘True’ or ‘False’, so I have one setting for the whole build script. Though a boolean parameter would be nice it seems I can only pass in strings as build variables, so I do the conversion to a boolean inside the script.

                          -TreatStyleCopViolationsErrorsAsWarnings $(ViolationsAsWarnings)

    image

Step 6 - Running the build

My test solution has two projects, with different settings.stylecop files. Once the new step was added to my build I could queue a build, by altering $(ViolationsAsWarnings)  variable I could make the build pass for fail.

        

image

       image

The detailed StyleCop result are available in the build log and are also placed in the drops folder in an XML format.

Note: One strange behaviour is that when you test the script outside TFS build you get a .XML and .LOG file for each project scanned. In TFS build you only see the .XML file in the drops folder, this I think is because the .LOG has been redirected into the main TFS vNext build logs.

Summary

So now I have a way to run StyleCop within a TFS vNext build.

Using these techniques there are no end of tools that can be wired into the build process, and I must say it is far easier than the TFS 2010, 2012, 2013 style workflow customisation.

Cross platform build with TFS 2015 vNext Build

I have been preparing for my Techorama session on TFS vNext build. One of the demo’s I am planning is to use the Node based cross platform build agent to build something on a Linux VM. Turns out this takes a few undocumented steps to get this going with the CTP of TFS 2015

The process I followed was:

  • I installed a Mint 17 VM
  • On the VM, I installed the Node VSOAgent as detailed in the npm documentation (or I could have built it from from source from GitHub to get the bleeding edge version)
  • I created a new agent instance
             vsoagent-installer
  • I then tried to run the configuration, but hit a couple of issues
              node vsoagent

URL error

The first problem was I was told the URL I provided was invalid. I had tried the URL of my local TFS 2015 CTP VM

http://typhoontfs:8080/tfs

The issue is that the vsoagent was initially developed for VSO and is expecting a fully qualified URL. To get around this, as I was on a local test network, I just added an entry to my Linux OS’s local /etc/hosts file, so I could call

http://typhoontfs.local:8080/tfs

This URL was accepted

401 Permissions Error

Once the URL was accepted, the next problem was I got a 401 permission error.

Now the release notes make it clear that you have to enable alternate credentials on your VSO account, but this is not a option for on premises TFS.

The solution is easy though (at least for a trial system). In IIS Manager on your TFS server enable basic authentication for the TFS application, you are warned this is not secure as passwords are sent in clear text, so probably not something to do on a production system

image

Once this was set the configuration of the client worked and I had an vsoagent running on my Linux client.

I could then go into the web based TFS Build.vNext interface and create a new empty build, adding the build tool I required, in my case Ant, using an Ant script stored with my project code in my TFS based Git repo.

When I ran the build it errored, as expected, my  Linux VM was missing all the build tools, but this was fixed by running apt-get on my Linux VM to install ant, ant-optional and the Java JDK. Obviously you need to install the tools you need.

So I have working demo, my Java application builds and resultant files dropped back into TFS. OK the configuration is not perfect at present, but from the GitHub site you can see the client  is being rapidly iterated

Build arguments are not returned for a build definition via the TFS API if they are left as default values

We use my TFS Alerts DSL to perform tasks when our TFS build complete, one of these is a job to increment the minor version number and reset the version start date (the value that generates third field – days since a point in time) if a build is set to the quality ‘release’ e.g. 1.2.99.[unique build id] where 99 is the days count since some past date could change to 1.3.0.[unique build id] (see this old post on how we do this in the build process)

I have just found a bug (feature?) in the way the DSL does this; turns out if you did not set the major and minor version argument values in the build editor (you just left them to their default values of 1 and 0) then the DSL fails as defaulted argument are not returned in the property set of the build definiation we process in the DSL. You would expect to get a 0 back, but you in fact get a null.

So if you have a build where you expect the version to increment and it does not, check the build definition and make sure the MajorVersion, MinorVersion (or whatever you called them) and version start date are all in bold

 

clip_image002

I have updated the code on Codeplex so that it gives a better error message in the event log if problem occurs with a build.

Fix for cannot run Windows 8.1 units test on a TFS 2013 Build Agent

I recently hit a problem that on one of our TFS 2013 build agents we could not run Windows 8.1 unit tests. Now as we know the build agent needs some care and attention to build Windows 8.1 at all, but we had followed this process. However, we still saw the issue that the project compiled but the tests failed with the error

Unit tests for Windows Store apps cannot be run with Limited User Account disabled. Enable it to run tests.’

image

I checked UAC settings and the build accounts rights (it ran as a local admin) all to no effect.

The answer it seems, thanks to the product group for the pointer, is that you have to make sure of the registry setting

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

"EnableLUA" =  1

On my failing VM this was set to zero.

I then had to reboot the the VM and also delete all contents of the c:\builds folder on my VM as due to the chance in UAC setting these old files had become read only to the build process.

Once this was all done my Windows 8.1 builds work correctly. Hope this post saves some other people some time

Setting a build version in a JAR file from TFS build

Whilst helping a Java based team (part of larger organisation that used many sets of both Microsoft and non-Microsoft tools) to migrate from Subversion to TFS I had to tackle their Jenkins/Ant based builds.

They could have stayed on Jenkins and switched to the TFS source provider, but they wanted to at least look at how TFS build would better allow them to  trace their builds against TFS work items.

All went well, we setup a build controller and agent specifically for their team and installed Java onto it as well the TFS build extensions. We were very quickly able to get our test Java project building on the new build system.

One feature that their old Ant scripts used was to store the build name/number into the Manifest of any JAR files created, a good plan as it is always good to know where something came from.

When asked as how to do this with TFS build I thought ‘no problem I will just use TFS build environment variable’ and add something like the following

<property environment="env"/>

<target name="jar">
        <jar destfile="${basedir}/javasample.jar" basedir="${basedir}/bin">
            <manifest>
                <attribute name="Implementation-Version" value="${env.TF_BUILD_BUILDNUMBER}" />
            </manifest>   
        </jar>
</target>

But this did not work, I just saw the text ${env.TF_BUILD_BUILDNUMBER}" in my manifest, basically the environment variable could not be resolved.

After a bit more of think I realised the problem is that the Ant/Maven build extensions for TFS are based on TFS 2008 style builds, the build environment variables are a TFS 2012 and later feature, so of course they are not set.

A quick look in the automatically generated TFSBuild.proj file generated for the build showed that the MSBuild $(BuildNumber) was passed into the Ant script as a property, so it could be referenced in the Ant Jar target (note the brackets change from () to {})

<target name="jar">
        <jar destfile="${basedir}/javasmaple.jar" basedir="${basedir}/bin">
            <manifest>
                <attribute name="Implementation-Version" value="${BuildNumber}" />
            </manifest>   
        </jar>
</target>

 

Once this change was made I then got the manifest I expected including the build number

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.4
Created-By: 1.8.0_25-b18 (Oracle Corporation)
Implementation-Version: JavaSample.Ant.Manual_20141216.7

Can’t build SSDT projects in a TFS build

Whilst building a new TFS build agent VM using our standard scripts I hit a problem that SSDT projects would not build, but was fine on our existing agents. The error was

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets (513): The "SqlBuildTask" task failed unexpectedly.
System.MethodAccessException: Attempt by method 'Microsoft.Data.Tools.Schema.Sql.Build.SqlTaskHost.OnCreateCustomSchemaData(System.String, System.Collections.Generic.Dictionary`2<System.String,System.String>)' to access method 'Microsoft.Data.Tools.Components.Diagnostics.SqlTracer.ShouldTrace(System.Diagnostics.TraceEventType)' failed.

The problem was fixed by doing an update via the Visual Studio > Tools > Extensions and Updates. Once this was completed the build was fine.

Seems there may have been an issue with the Update 3 generation SSDT tools, so older and newer versions seem OK. Our existing agents had already been patched.

Visual Studio crashes when trying to add an item to a TFS build workflow

There has for a long time been an issue that when you try to add a new activity to the toolbox when editing a TFS build workflow Visual Studio can crash. I have seen it many times and never got to the bottom of it. It seems to be machine specific, as one machine can work while another supposedly identical will fail, but I could never track down the issue.

Today I was on a machine that was failing, but …

But I found a workaround in a really old forum post. The workaround is to load the IDE from the command line with the /safemode flag

C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe /safemode

Once you do this you can edit the contents of your toolbox without crashes, and also your template if you wish. The best part is that once you exit the IDE and reload it as normal your new toolbox contents are still there.

Not perfect, but a good workaround

Cannot build a SSRS project in TFS build due to expired license

If you want to get your TFS build process to product SSRS RDL files you need to call the vsDevEnv custom activity to run Visual Studio (just like for SSIS packages). On our new TFS2013.3 based build agents this step started to fail, turns out the issue was not incorrect versions of DLLs or a some badly applied update, but that the license for Visual Studio on the build agent had expire.

I found it by looking at diagnostic logs in the TFS build web UI.

image

To be able to build BI project with Visual Studio you do need a licensed copy of Visual Studio on the build agent. You can use a trial license, but it will expire. Also remember if you license VS by logging in with your MSDN Live ID that too needs to be refreshed from time to time (that is what go me), so better to use a product key.