But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Running StyleCop from the command line and in a TFS 2015 vNext build

Virtually any automated build will require some customisation beyond a basic compile. So as part of my upcoming Techorama session on TFS 2015 vNext build I need a demo of using a custom script as part of the build process. Two common customisations we use are version stamping of assemblies and running code analysis tools. For vNext build there is already a sample of version stamping, so I thought getting StyleCop running would be a good sample.

The problem

Customisation in vNext build is based around running a script, in the case of a Windows based build agents this a PowerShell script. The problem with StyleCop is that it does not provide a command line iterface. The  StyleCop CodePlex project provides only a Visual Studio add-in. There is also the ALM Ranger’s TFS community custom build  activity, but I could find no current command line interface projects.

So I needed to build one.

Step 1 – Create a command line

So my first step was to create a command line version of StyleCop. I chose to use the community build activity as a starting point. I had planned to do this all in PowerShell, but quickly found that the conversion of parameter object types and the handling of the events StyleCop uses was a bit messy. So I decided to write a wrapper class in C# that presented the same parameters as the old TFS build activity, basically take the old code and remove the Windows Workflow logic. I then provided a Main (args) method to expose the object to the command line such that it was easy to provide the required parameters.

This can all be found on my GitHub site.

Note on solution structure: As I wanted this to work for PowerShell and the command prompt I had to place the Main(args[]) method .EXE entry point in a project that built an EXE and all the rest of the wrapper code in one that built a .DLL. This is because you cannot load a type in PowerShell using add-type from an assembly built as an EXE, you get a EXTENSION_NOT_SUPPORTED exception. It means there are two projects (a DLL and an EXE) when I would really have like a single one (the EXE)

So I now had a command line I could call from my PowerShell script

StyleCopCmdLine --f="File1.cs" "File2.cs" --s="AllSettingsEnabled.StyleCop"

A good starting point. However,  more a TFS build it makes more sense to call StyleCop directly in the PowerShell, why shell out to a command prompt to run an EXE when your can run the code directly in PowerShell?

Step 2 – Create a simple PowerShell script

The PowerShell required to run StyleCop using the wrapper is simple, just providing the same parameters as used for the EXE.

Add-Type -Path "StyleCopWrapper.dll"

 

$scanner = new-object StyleCopWrapper.Wrapper
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.XmlOutputFile = "$pwd\out.xml"
$scanner.LogFile = "$pwd\log.txt"
$scanner.SourceFiles =  @("file1.cs", "file2.cs") )
$scanner.SettingsFile = "settings.stylecop"
$scanner.AdditionalAddInPaths = @("C:\Program Files (x86)\StyleCop 4.7" )
$scanner.TreatViolationsErrorsAsWarnings = $false

$scanner.Scan()

write-host ("Succeeded [{0}]" -f $scanner.Succeeded)
write-host ("Violation count [{0}]" -f $scanner.ViolationCount)

See the GitHub site’s WIKI for the usage details.

Step 3 – Create a vNext build PowerShell script

So now we have the basic tools we need to run StyleCop from a TFS vNext build, but we do need a more complex script.

The script you use is up to you, mine looks for .csproj files and runs StyleCop recursively from the directories containing the .csproj files. This means I can have a different  setting.stylecop file for each project. In general I have more strict rules on production code than unit test e.g. for unit tests I am not bother about the XML method documentation, but for production code I make sure they are present and match the method parameters.

Note: As the script just uses parameters and environment variable it is easy to test outside TFS build, a great improvement over the old build system

#
# Script to allow StyleCop to be run as part of the TFS vNext build
#
[CmdletBinding()]
param
(
    # We have to pass this boolean flag as string, we cast it before we use it
    # have to use 0 or 1, true or false
    [string]$TreatStyleCopViolationsErrorsAsWarnings = 'False'
)

# local test values, should be commented out in production
#$Env:BUILD_STAGINGDIRECTORY = "C:\drops"
#$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"

if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_STAGINGDIRECTORY))
{
    Write-Error "You must set the following environment variables"
    Write-Error "to test this script interactively."
    Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY = "C:\drops"'
    exit 1
}

# pickup the build locations from the environment
$stagingfolder = $Env:BUILD_STAGINGDIRECTORY
$sourcefolder = $Env:BUILD_SOURCESDIRECTORY

# have to convert the string flag to a boolean
$treatViolationsErrorsAsWarnings = [System.Convert]::ToBoolean($TreatStyleCopViolationsErrorsAsWarnings)

Write-Host ("Source folder (`$Env)  [{0}]" -f $sourcefolder) -ForegroundColor Green
Write-Host ("Staging folder (`$Env) [{0}]" -f $stagingfolder) -ForegroundColor Green
Write-Host ("Treat violations as warnings (Param) [{0}]" -f $treatViolationsErrorsAsWarnings) -ForegroundColor Green
 
# the overall results across all sub scans
$overallSuccess = $true
$projectsScanned = 0
$totalViolations = 0


# load the StyleCop classes, this assumes that the StyleCop.DLL, StyleCop.Csharp.DLL,
# StyleCop.Csharp.rules.DLL in the same folder as the StyleCopWrapper.dll
Add-Type -Path "StyleCop\StyleCopWrapper.dll"
$scanner = new-object StyleCopWrapper.Wrapper

# Set the common scan options,
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.AdditionalAddInPaths = @($pwd) # in in local path as we place stylecop.csharp.rules.dll here
$scanner.TreatViolationsErrorsAsWarnings = $treatViolationsErrorsAsWarnings

# look for .csproj files
foreach ($projfile in Get-ChildItem $sourcefolder -Filter *.csproj -Recurse)
{
   write-host ("Processing the folder [{0}]" -f $projfile.Directory)

   # find a set of rules closest to the .csproj file
   $settings = Join-Path -path $projfile.Directory -childpath "settings.stylecop"
   if (Test-Path $settings)
   {
        write-host "Using found settings.stylecop file same folder as .csproj file"
        $scanner.SettingsFile = $settings
   }  else
   {
       $settings = Join-Path -path $sourcefolder -childpath "settings.stylecop"
       if (Test-Path $settings)
       {
            write-host "Using settings.stylecop file in solution folder"
            $scanner.SettingsFile = $settings
       } else
       {
            write-host "Cannot find a local settings.stylecop file, using default rules"
            $scanner.SettingsFile = "." # we have to pass something as this is a required param
       }
   }

   $scanner.SourceFiles =  @($projfile.Directory)
   $scanner.XmlOutputFile = (join-path $stagingfolder $projfile.BaseName) +".stylecop.xml"
   $scanner.LogFile =  (join-path $stagingfolder $projfile.BaseName) +".stylecop.log"
   
   # Do the scan
   $scanner.Scan()

    # Display the results
    Write-Host ("`n")
    write-host ("Base folder`t[{0}]" -f $projfile.Directory) -ForegroundColor Green
    write-host ("Settings `t[{0}]" -f $scanner.SettingsFile) -ForegroundColor Green
    write-host ("Succeeded `t[{0}]" -f $scanner.Succeeded) -ForegroundColor Green
    write-host ("Violations `t[{0}]" -f $scanner.ViolationCount) -ForegroundColor Green
    Write-Host ("Log file `t[{0}]" -f $scanner.LogFile) -ForegroundColor Green
    Write-Host ("XML results`t[{0}]" -f $scanner.XmlOutputFile) -ForegroundColor Green

    $totalViolations += $scanner.ViolationCount
    $projectsScanned ++
   
    if ($scanner.Succeeded -eq $false)
    {
      # any failure fails the whole run
      $overallSuccess = $false
    }

}

# the output summary
Write-Host ("`n")
if ($overallSuccess -eq $false)
{
   Write-Error ("StyleCop found [{0}] violations across [{1}] projects" -f $totalViolations, $projectsScanned)
}
elseif ($totalViolations -gt 0 -and $treatViolationsErrorsAsWarnings -eq $true)
{
    Write-Warning ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned)
}
else
{
   Write-Host ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned) -ForegroundColor Green
}

Step 4 – Adding a the script to the repo

To use the script it needs (and any associated files) to be placed in your source control. In my case it meant I create a folder called StyleCop off the root of my TFS 2015 CTP’s Git repo and in it placed the following files

  • PowerShell.ps1 – my script file
  • StyleCop.dll – the main StyleCop assembly taken from c:\program files (x86)\StyleCop 4.7. By placing it here it means we don’t need to actually install StyleCop on the build machine
  • StyleCop.csharp.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCop.csharp.rules.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCopWrapper.dll – the wrapper assembly from my GitHub site

Step 5 – Adding the script to a build process

Once the script is in the repo adding a new step to a vNext build is easy.

  • In a browser select the Build.vNext menu options
  • The build explorer will be show, right click on the build you wish to add a step to and select edit
  • Press the ‘Add build step’ button. The list of steps will be show, pick PowerShell

    image
  • As the script is in the repo we can reference it in the new step. in my case I set the script file name to

                          StyleCop/PowerShell.ps1
  • My script takes one parameter, if we should treat StleCop violations as warnings, this is set as the script argument. Note I am using a build variable $(ViolationsAsWarnings) set to a string value ‘True’ or ‘False’, so I have one setting for the whole build script. Though a boolean parameter would be nice it seems I can only pass in strings as build variables, so I do the conversion to a boolean inside the script.

                          -TreatStyleCopViolationsErrorsAsWarnings $(ViolationsAsWarnings)

    image

Step 6 - Running the build

My test solution has two projects, with different settings.stylecop files. Once the new step was added to my build I could queue a build, by altering $(ViolationsAsWarnings)  variable I could make the build pass for fail.

        

image

       image

The detailed StyleCop result are available in the build log and are also placed in the drops folder in an XML format.

Note: One strange behaviour is that when you test the script outside TFS build you get a .XML and .LOG file for each project scanned. In TFS build you only see the .XML file in the drops folder, this I think is because the .LOG has been redirected into the main TFS vNext build logs.

Summary

So now I have a way to run StyleCop within a TFS vNext build.

Using these techniques there are no end of tools that can be wired into the build process, and I must say it is far easier than the TFS 2010, 2012, 2013 style workflow customisation.

Cross platform build with TFS 2015 vNext Build

I have been preparing for my Techorama session on TFS vNext build. One of the demo’s I am planning is to use the Node based cross platform build agent to build something on a Linux VM. Turns out this takes a few undocumented steps to get this going with the CTP of TFS 2015

The process I followed was:

  • I installed a Mint 17 VM
  • On the VM, I installed the Node VSOAgent as detailed in the npm documentation (or I could have built it from from source from GitHub to get the bleeding edge version)
  • I created a new agent instance
             vsoagent-installer
  • I then tried to run the configuration, but hit a couple of issues
              node vsoagent

URL error

The first problem was I was told the URL I provided was invalid. I had tried the URL of my local TFS 2015 CTP VM

http://typhoontfs:8080/tfs

The issue is that the vsoagent was initially developed for VSO and is expecting a fully qualified URL. To get around this, as I was on a local test network, I just added an entry to my Linux OS’s local /etc/hosts file, so I could call

http://typhoontfs.local:8080/tfs

This URL was accepted

401 Permissions Error

Once the URL was accepted, the next problem was I got a 401 permission error.

Now the release notes make it clear that you have to enable alternate credentials on your VSO account, but this is not a option for on premises TFS.

The solution is easy though (at least for a trial system). In IIS Manager on your TFS server enable basic authentication for the TFS application, you are warned this is not secure as passwords are sent in clear text, so probably not something to do on a production system

image

Once this was set the configuration of the client worked and I had an vsoagent running on my Linux client.

I could then go into the web based TFS Build.vNext interface and create a new empty build, adding the build tool I required, in my case Ant, using an Ant script stored with my project code in my TFS based Git repo.

When I ran the build it errored, as expected, my  Linux VM was missing all the build tools, but this was fixed by running apt-get on my Linux VM to install ant, ant-optional and the Java JDK. Obviously you need to install the tools you need.

So I have working demo, my Java application builds and resultant files dropped back into TFS. OK the configuration is not perfect at present, but from the GitHub site you can see the client  is being rapidly iterated

Build arguments are not returned for a build definition via the TFS API if they are left as default values

We use my TFS Alerts DSL to perform tasks when our TFS build complete, one of these is a job to increment the minor version number and reset the version start date (the value that generates third field – days since a point in time) if a build is set to the quality ‘release’ e.g. 1.2.99.[unique build id] where 99 is the days count since some past date could change to 1.3.0.[unique build id] (see this old post on how we do this in the build process)

I have just found a bug (feature?) in the way the DSL does this; turns out if you did not set the major and minor version argument values in the build editor (you just left them to their default values of 1 and 0) then the DSL fails as defaulted argument are not returned in the property set of the build definiation we process in the DSL. You would expect to get a 0 back, but you in fact get a null.

So if you have a build where you expect the version to increment and it does not, check the build definition and make sure the MajorVersion, MinorVersion (or whatever you called them) and version start date are all in bold

 

clip_image002

I have updated the code on Codeplex so that it gives a better error message in the event log if problem occurs with a build.

Wrong package location when reusing a Release Management component

Whilst setting up a new agent based deployment pipeline in Release Management I decided I to reuse an existing component as it already had the correct package location set and correct transforms for the MSDeploy package. Basically this pipeline was a new copy of an existing website with different branding (css file etc.), but the same configuration options.

I had just expected this work, but I kept getting ‘file not found’ errors when MSDeploy was run. On investigation I found that the package location for the component was wrong, it was the build drop root, not the sub folder I had specified.

image 

I have no idea why.

The fix was to copy the component, and use this copy in the pipeline. It is probably what I should have done anyway, as I expect this web site to diverge from original one, so I will need to edit the web.config transforms, but not something I thought I would have had to do now to get it working.

Fix for cannot run Windows 8.1 units test on a TFS 2013 Build Agent

I recently hit a problem that on one of our TFS 2013 build agents we could not run Windows 8.1 unit tests. Now as we know the build agent needs some care and attention to build Windows 8.1 at all, but we had followed this process. However, we still saw the issue that the project compiled but the tests failed with the error

Unit tests for Windows Store apps cannot be run with Limited User Account disabled. Enable it to run tests.’

image

I checked UAC settings and the build accounts rights (it ran as a local admin) all to no effect.

The answer it seems, thanks to the product group for the pointer, is that you have to make sure of the registry setting

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

"EnableLUA" =  1

On my failing VM this was set to zero.

I then had to reboot the the VM and also delete all contents of the c:\builds folder on my VM as due to the chance in UAC setting these old files had become read only to the build process.

Once this was all done my Windows 8.1 builds work correctly. Hope this post saves some other people some time

Fix for ‘An unexpected error occurred. Close the windows and try again’ error adding Azure subscription to Visual Studio Release Management Tools

In preparation for my Techdays session next month, I have been sorting demos using the various Release Management clients.

When I tried to create a release from within Visual Studio using the ‘Release Management tools for Visual Studio I found I could not add my Azure subscriptions. I saw the error ‘An unexpected error occurred. Close the windows and try again’

image

I could download and import the subscription file, it showed the available storage accounts, but when I pressed save I got the rather unhelpful error ‘Object reference not set to an instance of an object’

image

Turns out the issue was a simple one, rights. The LiveID I had signed into Visual Studio as had no rights for Release Management on the VSO account running the Release Management service, even though it was a TPC administrator.

It is easier to understand the problem in the Release Management client. When I tried to set the Release Management Server Url (RM > Administration > Settings) to the required VSO Url as the LiveID I was using in Visual Studio I got the nice clear error shown below.

image

The solution was in the Release Management client to use the LiveID of the VSO account owner. I could then connect the Url in the Release Management client and then add my previously failing LiveID as a user for the release service.

image

Once this was done I was able to use this original LiveID in Visual Studio without a problem.

VSO gets ISO 27001 Certification and European Model Clauses

If the reason you have not been using VSO was concern over where it is hosted, then last week Microsoft made an announcement that could ease some of your worries, or at least your legal departments.  VSO now has ISO 27001 Certification and European Model Clauses; for more details see Brian Harry’s blog.

This added to the fact that since the end of last October you have been able to choose to host your VSO instance in Europe could well make VSO a more compelling option for many organisations who don’t want to have their own TFS servers on premises

Speaking at Microsoft Techdays on the 5th of February

On the 5th of February I will be presenting at Microsoft’s online Techdays event. My session is entitled ‘How are you going to deploy that? A look at configuration as code’

“It does not matter what platform you are developing for, you are going to have to deploy your product in the end. Too often in the past the question of deployment has been an afterthought. But it need not be this way, there are tools available that can help with deployment of your code and importantly the provisioning the underlying systems they need too. In this session we will look at Visual Studio Release Management’s vNext release pipeline seeing how it can leverage Desired State Configuration to provision environments and deploy applications”

There an loads of great sessions spread cross the three days of this online conference, why not register for the event online now? I am sure there will be something of interest.

How to edit registered Release Management deployment agent IP addresses if a VMs IP address changes

I have posted in the past that we have a number of agent based deployments using Release Management 2013.4 that point to network isolated Lab Management environments. Over Christmas we did some maintenance on our underlying Hyper-V servers, so everything got fully stopped and restarted. When the network isolated environment were restarted their DHCP assigned IP addresses on our company domain all changed (maybe we should have had longer DHCP lease times set?)

image

Worst of all some were reused and were actually swapped between environments, so an IP address that used to connect to a Server1 in environment Lab1 could be assigned to Server2 in environment Lab2. So basically all our deployments failed, usually because there server could not connect to the agent, but sometimes because the wrong VM responded.

Now for general Lab Management operations this was not an issue; inside the environment nothing had changed, the network range was still 192.168.23.x and externally SCVMM, MTM and the Test Controllers all knew what is going on and sorted themselves out. The problem was the Release Management deployment agent’s registration with the Release Management server. As I detailed in my previous post you have manually register the agents using shadow accounts. This means they are registered with their IP address at the time of registration, it does not change if the VMs IP address is reassigned with DHCP. It is up to you to fix it.

But how?

And that is the problem, there is no way to edit the IP addresses of the registered server’s deployment agents inside the Release Management admin tool. The only option I could find would be deleted the registered server and re-add them, but this requires them to be removed from any release pipelines. Something I did not want to do, to much work when I just wanted to fix an IP address.

The solution I found was to edit the IPAddress column in the underlying Server table in the ReleaseManagement DB. I did this with SQL Management Studio, nothing special. The only thing to note is that you cannot have duplicate IP addresses, so they had to be edited in an order to avoid duplication, using a temporary IP address during the edit process as I shuffled addresses around.

image

Once this was done everything leapt into life. I did not even need to restart the Release Management Server, just press the refresh button on the Server tab and saw all the agents had reconnected.

image

So a good dirty fix, but something I would have hoped would have been easier if the tools provided a means to edit the IP addresses

Note: This problem is specific to agent based deployment in Release Management. If you are using vNext DSC based deployment to network isolated VMs are registered using their DNS names on the corporate LAN e.g. VSLM-1344-e7858e28-77cf-4163-b6ba-1df2e91bfcab.lab.blackmarble.co.uk so the problem does not occur