But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

MSDeploy Parameters.xml can only replace web.config values is a value is already set

If you are using a parameters.xml file to set value with MSDeploy I have just found a gotcha. You need some value in the web.config file, not just an empty XML tag, else the replacement fails. So to explain…

I had the following parameters.xml file, and use Release Management to replace the __TAG__ values at deployment time.

<parameters>
  <parameter name="Domain" description="Please enter the name of the domain" defaultvalue="__Domain__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Web.Properties.Settings/setting[@name='Domain']/value/text()" />
  </parameter>

  <parameter name="AdminGroups" description="Please enter the name of the admin group" defaultvalue="__AdminGroups__" tags="">
    <parameterentry kind="XmlFile" scope="\\web.config$" match="/configuration/applicationSettings/Web.Properties.Settings/setting[@name='AdminGroups']/value/text()" />
  </parameter>
</parameters>

If my web.config file (in the MSDeploy package to be transformed) was set to

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>Blackmarble</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value />
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

or

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>Blackmarble</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value></value>
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

only the Domain setting was set.

To get both set I had to have a value for each property, even though they were being reset at deployment.

<applicationSettings>
    <Web.Properties.Settings>
      <setting name="Domain" serializeAs="String">
        <value>DummyDomain</value>
      </setting>
      <setting name="AdminGroups" serializeAs="String">
        <value>DummyAdmins</value>
      </setting>
    </BlackMarble.ISS.Expenses.Web.Properties.Settings>
  </applicationSettings>

Never seen that one before.

Running StyleCop from the command line and in a TFS 2015 vNext build

Virtually any automated build will require some customisation beyond a basic compile. So as part of my upcoming Techorama session on TFS 2015 vNext build I need a demo of using a custom script as part of the build process. Two common customisations we use are version stamping of assemblies and running code analysis tools. For vNext build there is already a sample of version stamping, so I thought getting StyleCop running would be a good sample.

The problem

Customisation in vNext build is based around running a script, in the case of a Windows based build agents this a PowerShell script. The problem with StyleCop is that it does not provide a command line iterface. The  StyleCop CodePlex project provides only a Visual Studio add-in. There is also the ALM Ranger’s TFS community custom build  activity, but I could find no current command line interface projects.

So I needed to build one.

Step 1 – Create a command line

So my first step was to create a command line version of StyleCop. I chose to use the community build activity as a starting point. I had planned to do this all in PowerShell, but quickly found that the conversion of parameter object types and the handling of the events StyleCop uses was a bit messy. So I decided to write a wrapper class in C# that presented the same parameters as the old TFS build activity, basically take the old code and remove the Windows Workflow logic. I then provided a Main (args) method to expose the object to the command line such that it was easy to provide the required parameters.

This can all be found on my GitHub site.

Note on solution structure: As I wanted this to work for PowerShell and the command prompt I had to place the Main(args[]) method .EXE entry point in a project that built an EXE and all the rest of the wrapper code in one that built a .DLL. This is because you cannot load a type in PowerShell using add-type from an assembly built as an EXE, you get a EXTENSION_NOT_SUPPORTED exception. It means there are two projects (a DLL and an EXE) when I would really have like a single one (the EXE)

So I now had a command line I could call from my PowerShell script

StyleCopCmdLine --f="File1.cs" "File2.cs" --s="AllSettingsEnabled.StyleCop"

A good starting point. However,  more a TFS build it makes more sense to call StyleCop directly in the PowerShell, why shell out to a command prompt to run an EXE when your can run the code directly in PowerShell?

Step 2 – Create a simple PowerShell script

The PowerShell required to run StyleCop using the wrapper is simple, just providing the same parameters as used for the EXE.

Add-Type -Path "StyleCopWrapper.dll"

 

$scanner = new-object StyleCopWrapper.Wrapper
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.XmlOutputFile = "$pwd\out.xml"
$scanner.LogFile = "$pwd\log.txt"
$scanner.SourceFiles =  @("file1.cs", "file2.cs") )
$scanner.SettingsFile = "settings.stylecop"
$scanner.AdditionalAddInPaths = @("C:\Program Files (x86)\StyleCop 4.7" )
$scanner.TreatViolationsErrorsAsWarnings = $false

$scanner.Scan()

write-host ("Succeeded [{0}]" -f $scanner.Succeeded)
write-host ("Violation count [{0}]" -f $scanner.ViolationCount)

See the GitHub site’s WIKI for the usage details.

Step 3 – Create a vNext build PowerShell script

So now we have the basic tools we need to run StyleCop from a TFS vNext build, but we do need a more complex script.

The script you use is up to you, mine looks for .csproj files and runs StyleCop recursively from the directories containing the .csproj files. This means I can have a different  setting.stylecop file for each project. In general I have more strict rules on production code than unit test e.g. for unit tests I am not bother about the XML method documentation, but for production code I make sure they are present and match the method parameters.

Note: As the script just uses parameters and environment variable it is easy to test outside TFS build, a great improvement over the old build system

#
# Script to allow StyleCop to be run as part of the TFS vNext build
#
[CmdletBinding()]
param
(
    # We have to pass this boolean flag as string, we cast it before we use it
    # have to use 0 or 1, true or false
    [string]$TreatStyleCopViolationsErrorsAsWarnings = 'False'
)

# local test values, should be commented out in production
#$Env:BUILD_STAGINGDIRECTORY = "C:\drops"
#$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"

if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_STAGINGDIRECTORY))
{
    Write-Error "You must set the following environment variables"
    Write-Error "to test this script interactively."
    Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY = "C:\drops"'
    exit 1
}

# pickup the build locations from the environment
$stagingfolder = $Env:BUILD_STAGINGDIRECTORY
$sourcefolder = $Env:BUILD_SOURCESDIRECTORY

# have to convert the string flag to a boolean
$treatViolationsErrorsAsWarnings = [System.Convert]::ToBoolean($TreatStyleCopViolationsErrorsAsWarnings)

Write-Host ("Source folder (`$Env)  [{0}]" -f $sourcefolder) -ForegroundColor Green
Write-Host ("Staging folder (`$Env) [{0}]" -f $stagingfolder) -ForegroundColor Green
Write-Host ("Treat violations as warnings (Param) [{0}]" -f $treatViolationsErrorsAsWarnings) -ForegroundColor Green
 
# the overall results across all sub scans
$overallSuccess = $true
$projectsScanned = 0
$totalViolations = 0


# load the StyleCop classes, this assumes that the StyleCop.DLL, StyleCop.Csharp.DLL,
# StyleCop.Csharp.rules.DLL in the same folder as the StyleCopWrapper.dll
Add-Type -Path "StyleCop\StyleCopWrapper.dll"
$scanner = new-object StyleCopWrapper.Wrapper

# Set the common scan options,
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.AdditionalAddInPaths = @($pwd) # in in local path as we place stylecop.csharp.rules.dll here
$scanner.TreatViolationsErrorsAsWarnings = $treatViolationsErrorsAsWarnings

# look for .csproj files
foreach ($projfile in Get-ChildItem $sourcefolder -Filter *.csproj -Recurse)
{
   write-host ("Processing the folder [{0}]" -f $projfile.Directory)

   # find a set of rules closest to the .csproj file
   $settings = Join-Path -path $projfile.Directory -childpath "settings.stylecop"
   if (Test-Path $settings)
   {
        write-host "Using found settings.stylecop file same folder as .csproj file"
        $scanner.SettingsFile = $settings
   }  else
   {
       $settings = Join-Path -path $sourcefolder -childpath "settings.stylecop"
       if (Test-Path $settings)
       {
            write-host "Using settings.stylecop file in solution folder"
            $scanner.SettingsFile = $settings
       } else
       {
            write-host "Cannot find a local settings.stylecop file, using default rules"
            $scanner.SettingsFile = "." # we have to pass something as this is a required param
       }
   }

   $scanner.SourceFiles =  @($projfile.Directory)
   $scanner.XmlOutputFile = (join-path $stagingfolder $projfile.BaseName) +".stylecop.xml"
   $scanner.LogFile =  (join-path $stagingfolder $projfile.BaseName) +".stylecop.log"
   
   # Do the scan
   $scanner.Scan()

    # Display the results
    Write-Host ("`n")
    write-host ("Base folder`t[{0}]" -f $projfile.Directory) -ForegroundColor Green
    write-host ("Settings `t[{0}]" -f $scanner.SettingsFile) -ForegroundColor Green
    write-host ("Succeeded `t[{0}]" -f $scanner.Succeeded) -ForegroundColor Green
    write-host ("Violations `t[{0}]" -f $scanner.ViolationCount) -ForegroundColor Green
    Write-Host ("Log file `t[{0}]" -f $scanner.LogFile) -ForegroundColor Green
    Write-Host ("XML results`t[{0}]" -f $scanner.XmlOutputFile) -ForegroundColor Green

    $totalViolations += $scanner.ViolationCount
    $projectsScanned ++
   
    if ($scanner.Succeeded -eq $false)
    {
      # any failure fails the whole run
      $overallSuccess = $false
    }

}

# the output summary
Write-Host ("`n")
if ($overallSuccess -eq $false)
{
   Write-Error ("StyleCop found [{0}] violations across [{1}] projects" -f $totalViolations, $projectsScanned)
}
elseif ($totalViolations -gt 0 -and $treatViolationsErrorsAsWarnings -eq $true)
{
    Write-Warning ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned)
}
else
{
   Write-Host ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned) -ForegroundColor Green
}

Step 4 – Adding a the script to the repo

To use the script it needs (and any associated files) to be placed in your source control. In my case it meant I create a folder called StyleCop off the root of my TFS 2015 CTP’s Git repo and in it placed the following files

  • PowerShell.ps1 – my script file
  • StyleCop.dll – the main StyleCop assembly taken from c:\program files (x86)\StyleCop 4.7. By placing it here it means we don’t need to actually install StyleCop on the build machine
  • StyleCop.csharp.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCop.csharp.rules.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCopWrapper.dll – the wrapper assembly from my GitHub site

Step 5 – Adding the script to a build process

Once the script is in the repo adding a new step to a vNext build is easy.

  • In a browser select the Build.vNext menu options
  • The build explorer will be show, right click on the build you wish to add a step to and select edit
  • Press the ‘Add build step’ button. The list of steps will be show, pick PowerShell

    image
  • As the script is in the repo we can reference it in the new step. in my case I set the script file name to

                          StyleCop/PowerShell.ps1
  • My script takes one parameter, if we should treat StleCop violations as warnings, this is set as the script argument. Note I am using a build variable $(ViolationsAsWarnings) set to a string value ‘True’ or ‘False’, so I have one setting for the whole build script. Though a boolean parameter would be nice it seems I can only pass in strings as build variables, so I do the conversion to a boolean inside the script.

                          -TreatStyleCopViolationsErrorsAsWarnings $(ViolationsAsWarnings)

    image

Step 6 - Running the build

My test solution has two projects, with different settings.stylecop files. Once the new step was added to my build I could queue a build, by altering $(ViolationsAsWarnings)  variable I could make the build pass for fail.

        

image

       image

The detailed StyleCop result are available in the build log and are also placed in the drops folder in an XML format.

Note: One strange behaviour is that when you test the script outside TFS build you get a .XML and .LOG file for each project scanned. In TFS build you only see the .XML file in the drops folder, this I think is because the .LOG has been redirected into the main TFS vNext build logs.

Summary

So now I have a way to run StyleCop within a TFS vNext build.

Using these techniques there are no end of tools that can be wired into the build process, and I must say it is far easier than the TFS 2010, 2012, 2013 style workflow customisation.

Cross platform build with TFS 2015 vNext Build

I have been preparing for my Techorama session on TFS vNext build. One of the demo’s I am planning is to use the Node based cross platform build agent to build something on a Linux VM. Turns out this takes a few undocumented steps to get this going with the CTP of TFS 2015

The process I followed was:

  • I installed a Mint 17 VM
  • On the VM, I installed the Node VSOAgent as detailed in the npm documentation (or I could have built it from from source from GitHub to get the bleeding edge version)
  • I created a new agent instance
             vsoagent-installer
  • I then tried to run the configuration, but hit a couple of issues
              node vsoagent

URL error

The first problem was I was told the URL I provided was invalid. I had tried the URL of my local TFS 2015 CTP VM

http://typhoontfs:8080/tfs

The issue is that the vsoagent was initially developed for VSO and is expecting a fully qualified URL. To get around this, as I was on a local test network, I just added an entry to my Linux OS’s local /etc/hosts file, so I could call

http://typhoontfs.local:8080/tfs

This URL was accepted

401 Permissions Error

Once the URL was accepted, the next problem was I got a 401 permission error.

Now the release notes make it clear that you have to enable alternate credentials on your VSO account, but this is not a option for on premises TFS.

The solution is easy though (at least for a trial system). In IIS Manager on your TFS server enable basic authentication for the TFS application, you are warned this is not secure as passwords are sent in clear text, so probably not something to do on a production system

image

Once this was set the configuration of the client worked and I had an vsoagent running on my Linux client.

I could then go into the web based TFS Build.vNext interface and create a new empty build, adding the build tool I required, in my case Ant, using an Ant script stored with my project code in my TFS based Git repo.

When I ran the build it errored, as expected, my  Linux VM was missing all the build tools, but this was fixed by running apt-get on my Linux VM to install ant, ant-optional and the Java JDK. Obviously you need to install the tools you need.

So I have working demo, my Java application builds and resultant files dropped back into TFS. OK the configuration is not perfect at present, but from the GitHub site you can see the client  is being rapidly iterated

Build arguments are not returned for a build definition via the TFS API if they are left as default values

We use my TFS Alerts DSL to perform tasks when our TFS build complete, one of these is a job to increment the minor version number and reset the version start date (the value that generates third field – days since a point in time) if a build is set to the quality ‘release’ e.g. 1.2.99.[unique build id] where 99 is the days count since some past date could change to 1.3.0.[unique build id] (see this old post on how we do this in the build process)

I have just found a bug (feature?) in the way the DSL does this; turns out if you did not set the major and minor version argument values in the build editor (you just left them to their default values of 1 and 0) then the DSL fails as defaulted argument are not returned in the property set of the build definiation we process in the DSL. You would expect to get a 0 back, but you in fact get a null.

So if you have a build where you expect the version to increment and it does not, check the build definition and make sure the MajorVersion, MinorVersion (or whatever you called them) and version start date are all in bold

 

clip_image002

I have updated the code on Codeplex so that it gives a better error message in the event log if problem occurs with a build.

Fix for timeout exporting a SQL Azure DB using PowerShell or SQLPackage.exe

I have been trying to export a SQL Azure DB as a .BACPAC using the command line

"C:\Program Files (x86)\Microsoft SQL Server\120\DAC\bin\SqlPackage.exe"
                              /action:Export
                             /sourceservername:myserver.database.windows.net
                             /sourcedatabasename:websitecontentdb
                             /sourceuser:sa@myserver /sourcepassword:<password> /targetfile:db.bacpac

The problem is the command times out after around an hour, at the ‘Extracting schema from database’ stage.

I got exactly the same issue if I use PowerShell as discussed in Sandrino Di Mattia’s post.

The issue is the Azure service  tier level I am running the SQL DB on.

image

If it is set to basic I get the error, if it is set to standard (even at the lowest settings) it works, and in my case the backup takes a couple of minutes.

I have seen similar problem trying to deploy a DACPAC to SQL Azure, and as I said in that post

‘Now the S0 instance is just over 2x the cost of a Basic , so if I was really penny pinching I could consider moving it back to Basic now the deployment is done.’

So the choice is mine, change the tier each time I want a export, or pay the extra cost

Wrong package location when reusing a Release Management component

Whilst setting up a new agent based deployment pipeline in Release Management I decided I to reuse an existing component as it already had the correct package location set and correct transforms for the MSDeploy package. Basically this pipeline was a new copy of an existing website with different branding (css file etc.), but the same configuration options.

I had just expected this work, but I kept getting ‘file not found’ errors when MSDeploy was run. On investigation I found that the package location for the component was wrong, it was the build drop root, not the sub folder I had specified.

image 

I have no idea why.

The fix was to copy the component, and use this copy in the pipeline. It is probably what I should have done anyway, as I expect this web site to diverge from original one, so I will need to edit the web.config transforms, but not something I thought I would have had to do now to get it working.

Fix for cannot run Windows 8.1 units test on a TFS 2013 Build Agent

I recently hit a problem that on one of our TFS 2013 build agents we could not run Windows 8.1 unit tests. Now as we know the build agent needs some care and attention to build Windows 8.1 at all, but we had followed this process. However, we still saw the issue that the project compiled but the tests failed with the error

Unit tests for Windows Store apps cannot be run with Limited User Account disabled. Enable it to run tests.’

image

I checked UAC settings and the build accounts rights (it ran as a local admin) all to no effect.

The answer it seems, thanks to the product group for the pointer, is that you have to make sure of the registry setting

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

"EnableLUA" =  1

On my failing VM this was set to zero.

I then had to reboot the the VM and also delete all contents of the c:\builds folder on my VM as due to the chance in UAC setting these old files had become read only to the build process.

Once this was all done my Windows 8.1 builds work correctly. Hope this post saves some other people some time

Living with a DD-WRT virtual router – three months and one day on (static DHCP leases)

Updated 28 Feb 2015 – Added bit on static addresses

image

When using a DD-WRT virtual router, I have realised it is worth setting static a MAC address in Hyper-V and DHCP lease on the router for any server  VMs  you want access to from your base system OS. In my case this is TFS demo VM a connect to all the time.

If you don’t do this the address of the VM seems to vary more than you might expect. So you keep having to edit the HOSTS file on your base OS to reference the VM by name.

You set the static MAC address in the Hyper-V setting

image

And the DHCP lease in the router Services tab, to make it a permanent lease leave the time  field empty

image

And finally the hosts file add an entry

# For the VM 00:15:5d:0b:27:05
192.168.1.99        typhoontfs

On down side of this is that if you are using snaphots as I am to address DHCP Wifi issues, you need to add the lease to any old snapshots you have, but once it is set there should be no more host file editing

Updated 28 Feb 2015

I have still found problems with strange routes in my routing table due to the internal switch issuing an address (and gateway) via DHCP; these seem to cause problems for my Microsoft direct access (a VPN) . Today I had the realisation I can avoid this problem by using a static address for my host PC’s connection to the internal router e.g. 192.168.1.50 set on the Windows adaptor, as opposed to DHCP. By making it static I avoid the issue of extra routes or DNS entries by simply not adding them.

image

Living with a DD-WRT virtual router – three months on

I posted in the past on my experience with DD-WRT router running in Hyper-V to allow my VMs internet access. A couple of months on I am still using it and I think have got around the worst of the issues.

The big problem is not with the DD-WRT router, but the way Hyper-V virtual switches use WiFi for some operating systems. Basically the summary is DHCP does not work for Linux VMs.

The best solution I have found to this problem is to use Hyper-V snapshots in which I hard code the correct IP settings for various networks, thus removing the need for DHCP.

At present I have three snapshots that I swap between as needed

image

  • One is set to use DHCP – I use this when my ‘external’ virtual switch is linked to a non-WIfi adaptor, usually the Ethernet in the office
  • One is hard coded for an IP address on my home router’s network, with suitable gateway and DNS setting
  • The final one is hard coded for my phone when it is being a Mifi

I can add  more as I need them, but as I find I am using hotel and client Wifi less and less as I am on an ‘all you can eat’ 4G mobile contract, I doubt I will need many more.

Seems to be working, i will report back if I learn more

When trying to load Office document from SharePoint I keep ending up in the Office Web Application

Whenever I tried to load an Office 2013 document from our SharePoint 2010 instance I kept ending up in the Office Web Application, the Office application was not being launched.

If I tried the use the ‘Open in Word’ context menu I got the following error (and before you ask yes I was in IE, IE11 in fact, and Office 2013 was installed)

image

My PC has been build of our standard System Center managed image, others using the same best image seemed OK, so what had gone wrong for me?

The launching of Office application features is managed by the ‘SharePoint OpenDocument Class’ IE add in (IE > Settings > Manage Add-ins). On my PC this whole add-in was missing, don’t know why.

image

The fix it turns out was to got into Control Panel > Add remove Programs > Office 2013 > Change and do a repair and a reboot. Once this was done Office launched as expected.