But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Running StyleCop from the command line and in a TFS 2015 vNext build

Virtually any automated build will require some customisation beyond a basic compile. So as part of my upcoming Techorama session on TFS 2015 vNext build I need a demo of using a custom script as part of the build process. Two common customisations we use are version stamping of assemblies and running code analysis tools. For vNext build there is already a sample of version stamping, so I thought getting StyleCop running would be a good sample.

The problem

Customisation in vNext build is based around running a script, in the case of a Windows based build agents this a PowerShell script. The problem with StyleCop is that it does not provide a command line iterface. The  StyleCop CodePlex project provides only a Visual Studio add-in. There is also the ALM Ranger’s TFS community custom build  activity, but I could find no current command line interface projects.

So I needed to build one.

Step 1 – Create a command line

So my first step was to create a command line version of StyleCop. I chose to use the community build activity as a starting point. I had planned to do this all in PowerShell, but quickly found that the conversion of parameter object types and the handling of the events StyleCop uses was a bit messy. So I decided to write a wrapper class in C# that presented the same parameters as the old TFS build activity, basically take the old code and remove the Windows Workflow logic. I then provided a Main (args) method to expose the object to the command line such that it was easy to provide the required parameters.

This can all be found on my GitHub site.

Note on solution structure: As I wanted this to work for PowerShell and the command prompt I had to place the Main(args[]) method .EXE entry point in a project that built an EXE and all the rest of the wrapper code in one that built a .DLL. This is because you cannot load a type in PowerShell using add-type from an assembly built as an EXE, you get a EXTENSION_NOT_SUPPORTED exception. It means there are two projects (a DLL and an EXE) when I would really have like a single one (the EXE)

So I now had a command line I could call from my PowerShell script

StyleCopCmdLine --f="File1.cs" "File2.cs" --s="AllSettingsEnabled.StyleCop"

A good starting point. However,  more a TFS build it makes more sense to call StyleCop directly in the PowerShell, why shell out to a command prompt to run an EXE when your can run the code directly in PowerShell?

Step 2 – Create a simple PowerShell script

The PowerShell required to run StyleCop using the wrapper is simple, just providing the same parameters as used for the EXE.

Add-Type -Path "StyleCopWrapper.dll"

 

$scanner = new-object StyleCopWrapper.Wrapper
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.XmlOutputFile = "$pwd\out.xml"
$scanner.LogFile = "$pwd\log.txt"
$scanner.SourceFiles =  @("file1.cs", "file2.cs") )
$scanner.SettingsFile = "settings.stylecop"
$scanner.AdditionalAddInPaths = @("C:\Program Files (x86)\StyleCop 4.7" )
$scanner.TreatViolationsErrorsAsWarnings = $false

$scanner.Scan()

write-host ("Succeeded [{0}]" -f $scanner.Succeeded)
write-host ("Violation count [{0}]" -f $scanner.ViolationCount)

See the GitHub site’s WIKI for the usage details.

Step 3 – Create a vNext build PowerShell script

So now we have the basic tools we need to run StyleCop from a TFS vNext build, but we do need a more complex script.

The script you use is up to you, mine looks for .csproj files and runs StyleCop recursively from the directories containing the .csproj files. This means I can have a different  setting.stylecop file for each project. In general I have more strict rules on production code than unit test e.g. for unit tests I am not bother about the XML method documentation, but for production code I make sure they are present and match the method parameters.

Note: As the script just uses parameters and environment variable it is easy to test outside TFS build, a great improvement over the old build system

#
# Script to allow StyleCop to be run as part of the TFS vNext build
#
[CmdletBinding()]
param
(
    # We have to pass this boolean flag as string, we cast it before we use it
    # have to use 0 or 1, true or false
    [string]$TreatStyleCopViolationsErrorsAsWarnings = 'False'
)

# local test values, should be commented out in production
#$Env:BUILD_STAGINGDIRECTORY = "C:\drops"
#$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"

if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_STAGINGDIRECTORY))
{
    Write-Error "You must set the following environment variables"
    Write-Error "to test this script interactively."
    Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\MySolution"'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_STAGINGDIRECTORY = "C:\drops"'
    exit 1
}

# pickup the build locations from the environment
$stagingfolder = $Env:BUILD_STAGINGDIRECTORY
$sourcefolder = $Env:BUILD_SOURCESDIRECTORY

# have to convert the string flag to a boolean
$treatViolationsErrorsAsWarnings = [System.Convert]::ToBoolean($TreatStyleCopViolationsErrorsAsWarnings)

Write-Host ("Source folder (`$Env)  [{0}]" -f $sourcefolder) -ForegroundColor Green
Write-Host ("Staging folder (`$Env) [{0}]" -f $stagingfolder) -ForegroundColor Green
Write-Host ("Treat violations as warnings (Param) [{0}]" -f $treatViolationsErrorsAsWarnings) -ForegroundColor Green
 
# the overall results across all sub scans
$overallSuccess = $true
$projectsScanned = 0
$totalViolations = 0


# load the StyleCop classes, this assumes that the StyleCop.DLL, StyleCop.Csharp.DLL,
# StyleCop.Csharp.rules.DLL in the same folder as the StyleCopWrapper.dll
Add-Type -Path "StyleCop\StyleCopWrapper.dll"
$scanner = new-object StyleCopWrapper.Wrapper

# Set the common scan options,
$scanner.MaximumViolationCount = 1000
$scanner.ShowOutput = $true
$scanner.CacheResults = $false
$scanner.ForceFullAnalysis = $true
$scanner.AdditionalAddInPaths = @($pwd) # in in local path as we place stylecop.csharp.rules.dll here
$scanner.TreatViolationsErrorsAsWarnings = $treatViolationsErrorsAsWarnings

# look for .csproj files
foreach ($projfile in Get-ChildItem $sourcefolder -Filter *.csproj -Recurse)
{
   write-host ("Processing the folder [{0}]" -f $projfile.Directory)

   # find a set of rules closest to the .csproj file
   $settings = Join-Path -path $projfile.Directory -childpath "settings.stylecop"
   if (Test-Path $settings)
   {
        write-host "Using found settings.stylecop file same folder as .csproj file"
        $scanner.SettingsFile = $settings
   }  else
   {
       $settings = Join-Path -path $sourcefolder -childpath "settings.stylecop"
       if (Test-Path $settings)
       {
            write-host "Using settings.stylecop file in solution folder"
            $scanner.SettingsFile = $settings
       } else
       {
            write-host "Cannot find a local settings.stylecop file, using default rules"
            $scanner.SettingsFile = "." # we have to pass something as this is a required param
       }
   }

   $scanner.SourceFiles =  @($projfile.Directory)
   $scanner.XmlOutputFile = (join-path $stagingfolder $projfile.BaseName) +".stylecop.xml"
   $scanner.LogFile =  (join-path $stagingfolder $projfile.BaseName) +".stylecop.log"
   
   # Do the scan
   $scanner.Scan()

    # Display the results
    Write-Host ("`n")
    write-host ("Base folder`t[{0}]" -f $projfile.Directory) -ForegroundColor Green
    write-host ("Settings `t[{0}]" -f $scanner.SettingsFile) -ForegroundColor Green
    write-host ("Succeeded `t[{0}]" -f $scanner.Succeeded) -ForegroundColor Green
    write-host ("Violations `t[{0}]" -f $scanner.ViolationCount) -ForegroundColor Green
    Write-Host ("Log file `t[{0}]" -f $scanner.LogFile) -ForegroundColor Green
    Write-Host ("XML results`t[{0}]" -f $scanner.XmlOutputFile) -ForegroundColor Green

    $totalViolations += $scanner.ViolationCount
    $projectsScanned ++
   
    if ($scanner.Succeeded -eq $false)
    {
      # any failure fails the whole run
      $overallSuccess = $false
    }

}

# the output summary
Write-Host ("`n")
if ($overallSuccess -eq $false)
{
   Write-Error ("StyleCop found [{0}] violations across [{1}] projects" -f $totalViolations, $projectsScanned)
}
elseif ($totalViolations -gt 0 -and $treatViolationsErrorsAsWarnings -eq $true)
{
    Write-Warning ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned)
}
else
{
   Write-Host ("StyleCop found [{0}] violations warnings across [{1}] projects" -f $totalViolations, $projectsScanned) -ForegroundColor Green
}

Step 4 – Adding a the script to the repo

To use the script it needs (and any associated files) to be placed in your source control. In my case it meant I create a folder called StyleCop off the root of my TFS 2015 CTP’s Git repo and in it placed the following files

  • PowerShell.ps1 – my script file
  • StyleCop.dll – the main StyleCop assembly taken from c:\program files (x86)\StyleCop 4.7. By placing it here it means we don’t need to actually install StyleCop on the build machine
  • StyleCop.csharp.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCop.csharp.rules.dll – also from c:\program files (x86)\StyleCop 4.7
  • StyleCopWrapper.dll – the wrapper assembly from my GitHub site

Step 5 – Adding the script to a build process

Once the script is in the repo adding a new step to a vNext build is easy.

  • In a browser select the Build.vNext menu options
  • The build explorer will be show, right click on the build you wish to add a step to and select edit
  • Press the ‘Add build step’ button. The list of steps will be show, pick PowerShell

    image
  • As the script is in the repo we can reference it in the new step. in my case I set the script file name to

                          StyleCop/PowerShell.ps1
  • My script takes one parameter, if we should treat StleCop violations as warnings, this is set as the script argument. Note I am using a build variable $(ViolationsAsWarnings) set to a string value ‘True’ or ‘False’, so I have one setting for the whole build script. Though a boolean parameter would be nice it seems I can only pass in strings as build variables, so I do the conversion to a boolean inside the script.

                          -TreatStyleCopViolationsErrorsAsWarnings $(ViolationsAsWarnings)

    image

Step 6 - Running the build

My test solution has two projects, with different settings.stylecop files. Once the new step was added to my build I could queue a build, by altering $(ViolationsAsWarnings)  variable I could make the build pass for fail.

        

image

       image

The detailed StyleCop result are available in the build log and are also placed in the drops folder in an XML format.

Note: One strange behaviour is that when you test the script outside TFS build you get a .XML and .LOG file for each project scanned. In TFS build you only see the .XML file in the drops folder, this I think is because the .LOG has been redirected into the main TFS vNext build logs.

Summary

So now I have a way to run StyleCop within a TFS vNext build.

Using these techniques there are no end of tools that can be wired into the build process, and I must say it is far easier than the TFS 2010, 2012, 2013 style workflow customisation.

‘TF400499: You have not set your team field’ when trying to update Team Settings via the TFS API

I have recently been automating TFS admin processes such as creating a new team within an existing team project. The TFS team is now our primary means we use to segment work so we need to create new teams fairly often, so automation makes good sense.

As far as I can see, there are no command line tools, like TF.EXE or WITADMIN.EXE, to do most of the team operations. They are usually browser based. So I have been using PowerShell and the TFS API for the automation.

I hit the problem that when trying to save the team’s backlog iteration, iteration paths etc. for a newly created team using  the SetTeamSettings method. I was seeing

Exception calling "SetTeamSettings" with "2" argument(s): "TF400499: You have not set your team field."
At C:\Projects\tfs2012\TFS\PowerShell\Set-TfsTeamConfig.ps1:37 char:1
+ $configSvc.SetTeamSettings($configs[0].TeamId , $configs[0].TeamSettings)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : SoapException

This was strange as I had previously tested my PowerShell script file with hard coded values to update an existing team without error. When I inspecting the parameters being passed all looked OK, I had set a value for the team field and the read only property defining where to store the team data was correct (into the AreaPath).

After far to long I realised the problem was I had set the team field to the correct value e.g. ‘My project\My team’, but I had not created this area path before trying to reference it. Once I had created the required area path my scripted worked as expect.

So the TF400499 error is a little confusing, it does not mean ‘not set’ but ‘not set to a valid value’

When you think it cannot get worse...

... you end up presenting at TechEd.

Yesterday was fun (of a sort) I ended up doing the demo section of the ESB Guidance session at TechEd. This session was scheduled to be done by Robert Hogg (Black Marble) and Ewan Fairweather (Microsoft) but Ewan had to fly home early unexpectedly on Friday morning, so leaving somewhat of a gap.

So you say, 'that is not too bad you just step in and do the prepared and scripted demo'. Well in a perfect world you would, but about 6 hours before our session was on the formal release of  ESB 1.0 was posted to the MSDN site. As our demo was based on a CTP build, and as we knew the final release was somewhat different, we thought it only right to at least show the new documentation and file structures. So a hectic few hours were had by all.

I hope anyone attended the session go what they wanted out to if. I had to leave for my flight soon after the session so have heard no feedback other than the people who came to chat at the end of the session, who seemed happy. I am sure Robert will know more when he gets back, as his flight was later he had time to return to the olympian heights of the speaker lounge to once more feast upon unicorn steaks and ambrosia (well get coffee in a proper cup not a paper one at least).

As I made the mistake of not changing my TechEd speakers shirt before the flight home, not realizing the shirt made me look like a member of EasyJet staff, my outstanding question is - is there any future event in my life where the lovely grey/blue with orange trim TechEd speakers shirt is appropriate wear?

TechEd Barcelona - update

I have not been blogging much from here have I, it is not that the sessions are not that interesting, but no single item has been giving me an huge urge to write.

As I said in my last post I think this is a conference of best practice ideas and as such you tend to pick up a useful nugget here and there which you store away for future use. This is particularity relevant as at present I am reviewing our engineering process to improve our software development life cycle.

Like many companies we use a variety of tools beyond Visual Studio such as CruiseControl, nUnit and our own home grown work tracking system. I have to consider when it is advantageous to swap these for the new features in Visual Studio Team System 2008. Being pragmatic this is always going to be a slow migration, these is little point investing time in moving an old project that barely still under maintenance to a new system. In fact I have chosen to only move our active projects from our old SourceSafe based system to TFS at a major release point, e.g. V1 to V2, snap-shoting it at this point and not bothering to bring over all the change history.

The tools round the edge is another question. If you, as we do, have an investment in nUnit and CruiseControl for projects is there any good case to rework everything to MSTEST and TFS Build? In the long term I think the answer is yes, to get a unified end to end solution, but it is hard to justify the time to do an 'instant' swap over, so again it will be slow move. Especially when you can use a combined system e.g. have old testing nUnit and new ones in MSTEST pulling it all together with CruiseControl which can happily access the TFS SCC, build using MSBUILD and run all the testing frameworks.

 

Anyway Roy Osherove is tuning up his guitar for a session on testing to time to go....

No sizzle and not much sausage

The keynote at TechEd was as expected, we all knew about the impending release  of VS2008, and still no fixed date yet (so no sausage there) and not really anything announced product wise that was not already in the blog sphere (so no real sizzle).

I think this is going to be conference on delivering on last years promises; how to get the best from the tools and technology announced last year, now that they are now really production ready and the early adopters have had a year to play with them.

My suggestion to all attendees - check the sessions with the real world experience. Due to the CTP and beta programs, there are some real experts out there on these products and many of them are presenting here (or in the audiences at the number interactive sessions).

How to create a community

Just come out of an interesting set of round table events for 'community influencers' at TechEd. These are people who are active in both the online and face-to-face communities from all round Europe (and Australia - the reach of TechEd Europe!) attended.

In the sessions I went to the general discussion was on the point I posted about a few weeks ago and that had been a running conversions on a number of UK blogs. I was refeshing (or sad?) to find the problems we have seen at home over attendance are the same around Europe:

  • It is hard to get people to attend events in the evening
  • It is hard to convert attendees to active community members
  • A very small percentage of people who view online forums contribute.

As you would expect there is no single answer, and for most ideas there was someone to say 'we tried that and it did not work for us'. However, it did come out that things that fail for one group work for others - there is no silver bullet. So try anything and everything to get people engaged.

A general it was felt 'marketing presentations' do not draw people in, neither do events that cover what can be found on-line. Most people agreed that events, maybe in a panel or round table format, that provide real world experience or 'war stories' as I call them are often the ones that get the most interest. Of cause it helps if the speaker presents in an engaging style, but this is mitigated if you can get the whole room involved.

From my experience some of the most interesting community events  have been to are technology agnostic and focus on general development for project management issues, notably in a group workshop style. Such as those at the Extreme Programming club, but even with this interesting content this group has struggled for number. As I said before the fact I like technology agnostic groups, as a NET developer I know there is much I can learn from Java developers and vice-versa, does not mean that this is right for all.

There was an underlying discussion of how many people in the industry were looking to the community as a means to professional development, as opposed to IT being just a job that ended at 5pm. Moving the latter group into  being hard - can you engage people who have lost the 'joy for their career'?

I am sure this pre conference event will generate some online activity, keep an eye out for it.

If it is Monday it must be Barcelona

What an awful journey I have had since my last post, the trip back from the USA was fine; the problems started getting from the UK to Spain. Basically fog at Liverpool stopped all flights so we had to change airline and airport to get here in time for the TechEd conference. Should have been here by noon on Sunday and actually got here nearer 9pm, so no sigh seeing for me.

Anyway registered and at the conference venue now, so let it begin.....

Last day at SOA - day 4

Seems quiet here today at the SOA conference, less people about. I wonder how many have sneaked off early for flights? It does seem to be mostly Europeans left, judging by the languages I have heard.

This does not mean parking was easy this morning, as in the same conference center Bill Gates and Bill Clinton are speaking at an MSPAC event. Now a bit of google'ing shows MSPAC is either:

  1. Microsoft Political Action Committee 
  2. Resources for Ms. Pac-Man(www.mspac.com)
  3. Maple Syrup Producers Association of Connecticut

I really hope it is one the last two, but I doubt it.

I always thing the best thing you can bring away from conferences such as this are best practice and gotta's, to that end I saw a great session by Stephen Thomas on best practice with orchestrations, you can find his sample and slides on www.biztalkgurus.com

So that's it for the SOA conference, off to the airport now, about 12 hours at home in the UK then off to TechEd in Barcelona. Oh what a jet set life I lead.