But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

vNext Build editor filePath control always returns a path even if you did not set a value

You can use the filePath type in a vNext VSTS/TFS task as shown below 

{
     "name": "settingsFile",
     "type": "filePath",
     "label": "Settings File",
     "defaultValue": "",
     "required": false,
     "helpMarkDown": "Path to single settings files to use (as opposed to files in project folders)",
     "groupName":"advanced"
   }

to present a file picker dialog in the build editor that allows the build editor to pick a file or folder in the build’s source repository

image

While doing some task development recently I found that this control did not behave as I had expected

  • If a value is explicitally set then the full local path to selected file or folder (on the build agent) is returned e.g. c:\agent\_work\3\s\yourfolder\yourfile.txt – just as expected
  • If you do not set a value, or set a value then remove your setting when you edit a build, then you don’t get an empty string, as I had expected. You get the path to the BUILD_SOURCESDIRECTORY e.g. c:\agent\_work\3\s – makes sense when you think about it.

So, if as in my case, you wanted to have specific behaviour only when this values was set to something other than the repo root you need to add some guard code


if ($settingsFile -eq $Env:BUILD_SOURCESDIRECTORY )
{
    $settingsFile = ""
}

Once I did this my task behaved as a needed, only running the code when the user had set an explicit value for the settings file.

A VSTS vNext build task to run StyleCop

I have previously posted on how a PowerShell script can be used to run StyleCop as part of vNext VSTS/TFS build. Now I have more experience with vNext tasks it seemed a good time to convert this PowerShell script into a true task that can deploy StyleCop and making it far easier to expose the various parameters StyleCop allows.

To this end I have written a new StyleCop task that can be found in my vNext Build Repo, this has been built to use the 4.7.49.0 release of StyleCop (so you don’t need to install StyleCop in the build machine, so it works well on VSTS).

To use this task:

  1. Clone the repo
  2. Build the tasks using Gulp
  3. Upload the task you require to your VSTS or TFS instance

Once this is done you can add the task to your build. You probably won’t need to set any parameters as long as you have settings.stylecop files to define your StyleCop ruleset in the same folders as your .CSPROJ files (or are happy default rulesets).

If you do want to set parameters your options are:

  • TreatStyleCopViolationsErrorsAsWarnings - Treat StyleCop violations errors as warnings, if set to False any StyleCop violations will cause the build to fail (default false).

And on the advanced panel

  • MaximumViolationCount - Maximum violations before analysis stops (default 1000)
  • ShowOutput - Sets the flag so StyleCop scanner outputs progress to the console (default false)
  • CacheResults - Cache analysis results for reuse (default false)
  • ForceFullAnalysis - Force complete re-analysis (default true)
  • AdditionalAddInPath - Path to any custom rule sets folder, the directory cannot be a sub directory of current directory at runtime as this is automatically scanned. This folder must contain your custom DLL and the Stylecop.dll and Stylecop.csharp.cs else you will get load errors
  • SettingsFile - Path to single settings files to use for all analysis (as opposed to settings.stylecop files in project folders)

 

image

 

When you run the build with the new task you sould expect to see a summary of the StyleCop run on the right

image

A new vNext task to run StyleCop

Update 6 Feb 2016 - I have made some major changes to this task to expose more parameters, have a look at this post that details the newer version

Today a good way to pull together all your measures of code quality is to run SonarQube within your automated build; in a .NET world this can show changes in quality over time for tools such as FxCop (Code Analysis) and StyleCop. However sometime you might just want to run one of these tools alone as part of your automated build. For Code Analysis this is easy, it is built into Visual Studio just set it as a property on the project. For StyleCop it is a bit more awkward as StyleCop was not designed to be run from the command line.

To get around this limitation I wrote a command line wrapper that could be used within a build process, see my blog post for details of how this could be used with vNext build.

Well that was all best part of a year ago. Now I have more experience with vNext build it seems wrong to use just a PowerShell script when I could create a build task that also deploys StyleCop. I have eventually got around to writing the task which you can find in my vNextBuild repo.

Once the task is uploaded to your TFS for VSTS instance, the StyleCop task can be added into any build process. The task picks up the file locations from the build environment variables and then hunts for StyleCop settings files (as detailed in my previous post). The only argument that needs to be set is whether the buidl should fail if there are violations

 

image

Once this is all setup the build can be run and the violations will be shown in the build report, whether the build fails or passes is down to how you set the flag for the handling of violations

image

.

Nuget restore fails to restore all the files on VSTS build if using project.json files

We are currently working on updating a Windows 8 application to be a Windows 10 Universal application. This has caused a few problem on a TFS vNext automated build box. The revised solution builds fine of the developers box and fine on the build VM if opened in Visual Studio, but fails if built via the VSTS vNext build CI MSBuild process showing loads of references missing.

Turns out the issue was due to Nuget versions.

The problem was that as part of the upgrade the solution had gained some new projects. These used the new project.json file to manage their Nuget references, as opposed to the old packages.config file. Visual Studio 2015 handles these OK, hence the build always working in the IDE, but you need Nuget.exe 3.0 or later for it to handle the new format. The version of Nuget installed as part of my vNext build agent was 2.8. So no wonder it had a problem.

To test my assumptions I added a Nuget Installer task to my build and set an explicit path to the newest version of Nuget.

image

Once this was done my build was fine.

So my solution options are

  1. Don’t use the agents shipped with TFS 2015, get newer ones fro VSTS this has a current version of Nuget (just make sure your agent/server combination is supported. I have had issues with the latest VSTS agent and a TFS 2015 RTM instance)
  2. Manually replace the version of Nuget.exe in my build agent tools folder – easy to forget you did but works
  3. Place a copy of the version of Nuget.exe I want on each build VM and reference it s path explicitly (as I did to diagnose the problem)

The first option is the best choice as it is always a good plan to keep build agents up to date

Why you need to use vNext build tasks to share scripts between builds

Whilst doing a vNext build from a TFVC repository I needed map both my production code branch and a common folder of scripts that I intended to use in a number of builds, so my build workspace was set to

  • Map – $/BM/mycode/main                                     - my production code
  • Map – $/BM/BuildDefinations/vNextScripts  - my shared PowerShell I wish to run in different builds e.g. assembly versioning.

As I wanted this to be a CI build, I also  set the trigger to $/tp1/mycode/main

The problem I found was that with my workspace set as above, the associated changes for the build include anything checked into $/BM and below. Also the source branch was set as $/BM

image

 

To fix this problem I had to remove the mapping to the scripts folder, once this was done  the associated changes shown were only those for my production code area, and the source branch was listed correctly.

But what to do about running my script?

I don’t really want to have to copy a common script to each build, huge potential for error there, and versioning issues if I want a common script in all build. The best solution I found was to take the PowerShell script, in my case the sample assembly versioning script provided for VSO, and package it as a vNext build Task. It took no modification just required the addition of a manifest file. You can find the task on my vNextBuild Github repo

This custom task could then be uploaded to my TFS server and used in all my builds. As it picks up it variable from environment variables it required so configuration, extracting the version number for the build number format.

image

If you wish to use this task, you need to follow the same instructions to setup your development environment as for the VSO Tasks, then

  1. Clone the repo https://github.com/rfennell/vNextBuild.git
  2. In the root of the repo run gulp to build the task
  3. Use tfx to upload the task to your TFS instance

Versioning a VSIX package as part of the TFS vNext build (when the source is on GitHub)

I have recently added a CI build to my GitHub stored ParametersXmlAddin VSIX project. I did this using Visual Studio Online’s hosted build service, did you know that this could used to build source from GitHub?

As part of this build I wanted to version stamp the assemblies and the resultant VSIX package. To do the former I used the script documented on MSDN, for the latter I also used the same basic method of extracting the version from the build number as used in the script for versioning assemblies. You can find my VSIX script stored in this repo.

I added both of these scripts in my ParametersXmlAddin project repo’s Script folder and just call them at the start of my build with a pair of PowerShell tasks. As they both get the build number from the environment variables there is no need to pass any arguments.

image

I only wanted to publish the VSIX package. This was done by setting the contents filter on the Publish Build Artifacts task to **\*.vsix

image

The final step was to enable the badge for the build, this is done on the General tab. Once enabled, I copied the provided URL for the badge graphics that shows the build status and added this as an image to the Readme.MD file on my repo’s wiki

image

Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build

This article was first published on the Microsoft’s UK Developers site as Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build

With the advent of vNext build in TFS 2015 and Visual Studio Online running unit tests that are not MSTest based within your build process is far more straightforward than it used to be. No longer do you have to use custom XAML build activities or tell all your TFS build controllers where the test runner assemblies are. The ‘out the box’ vNext build Visual Studio Test task will automatically load any test adaptors it finds in the path specified for test runners in its advanced properties, a path that can be populated via NuGet.

Running nUnit tests

All this means that to find and run MSTest and nUnit tests as part of your build all you have to do is as follows

  1. Create a solution that contains a project with MStest and nUnit tests, in my sample this is a MVC web application project with its automatically created MSTest unit tests project.
  2. In the test project add some nUnit tests. Use NuGet to add the references to nUnit to the test project so it compiles.
  3. Historically in your local Visual Studio instance you needed to install the nUnit Test Runner VSIX package from Visual Studio Gallery – this allows Visual Studio to discover your nUnit tests, as well as any MSTest ones, and run them via the built in Test Explorer

    image

    IMPORTANT Change –
    However installing this VSIX package is no longer required. If you use Nuget to add the nUnit Test Runner to the solution, as well as the nUnit package itself, then Visual Studio can find the nUnit tests without the VSIX package. This is useful but not world changing on your development PC, but when on the build box it means the NuGet restore will make sure the nUnit test adapter assemblies are pulled down onto the local build boxes file system and used to find tests with no extra work.

    Note
    : If you still want to install the VSIX package on your local Visual Studio instance you can, it is just you don’t have to.
  4. Check in your solution into TFS/VSO source control. It does not matter if it is TFVC or Git based
  5. Create a new vNext build using the Visual Studio template
  6. You can leave most of the parameters on default setting. But you do need to edit the Visual Studio Test task’s advanced settings to point at the NuGet packages folder for your solution (which will be populated via NuGet restore) so the custom nUnit test adaptor can be found i.e. usually setting it to  $(Build.SourcesDirectory)\packages

    image
  7. The build should run and find your tests, the MStest ones because they are built in and the nUnit ones because it found the custom test adaptor due to the NuGet restore being done prior to the build. The test results can be found on the build summary page

    image

 

But what if you want run Jasmine.JS test?

If you want to run Jasmine JavaScript unit tests the process is basically the same. The only major difference is that you do still need to install the Chutzpah Test runner on your local Visual Studio as a VSIX package to run the tests locally. There is a NuGet package for the Chutzpah test runner so you can avoid having to manually unpack the VSIX and get it into source control to deploy it to the build host (unless you really want to follow this process), but this package does not currently enable Visual Studio to find the Jasmine tests without the VSIX extension being installed, or at least it didn’t for me.

Using the solution I used before

  1. Use NuGet to add Jasmine.JS to the test project
  2. Add a test file to the test project e.g. mycode.tests.js (adding any JavaScript references needed to find any script code under test in the main WebApp project)
  3. Install the Chutzpah Test runner in your local Visual Studio as a VSIX extension, restart Visual Studio
  4. You should now be able to see and run the Jasmine test run in the test runner as well as the MSTest and nUnit tests.

    image
  5. Add the NuGet package for the Chutzpah test runner to your solution, this is a solution level package, so does not need to be associated with any project.
  6. Check the revised code into source control
  7. In your vNext build add another Visual Studio Test task, set the test assembly to match your javascript test naming convention e.g. **\*.tests.js and the path to the custom test adaptor to $(Build.SourcesDirectory)\packages (as before)

    image
  8. Run the revised build.

    image
  9. You should see the two test tasks run and a pair of test results in the summary for the build.

So now hopefully you should find this a more straight forward way to added testing to your vNext builds. Allowing easy use of both your own build boxes and the hosted build service for VSO with testing frameworks they do not support ‘out the box’

Running Typemock Isolator based tests in TFS vNext build

Typemock Isolator provides a way to ‘mock the un-mockable’, such as sealed private classes in .NET, so can be a invaluable tool in unit testing. To allow this mocking Isolator interception has to be started before any unit tests are run and stopped when completed. For a developer this is done automatically within the Visual Studio IDE, but on build systems you have to run something to do this as part of your build process. Typemock provide documentation and tools for common build systems such as MSBuild, Jenkins, Team City and TFS XAML builds. However, they don’t provide tools or documentation on getting it working with TFS vNext build, so I had to write my own vNext build Task to do the job, wrapping Tmockrunner.exe provided by Typemock which handles the starting and stopping of mocking whilst calling any EXE of your choice.

tmockrunner <name of the test tool to run> <and parameters for the test tool>

Microsoft provide a vNext build task to run the vstest.console.exe. This task generates all the command line parameters needed depending on the arguments provided for the build task. The source for this can be found on any build VM (in the [build agent folder]\tasks folder after a build has run) or on Microsoft’s vso agent github repo. I decided to use this as my starting point, swapping the logic to generate the tmockrunner.exe command line as opposed to the one for vstest.console.exe. You can find my task on my github. It has been developed in the same manner as the Microsoft provided tasks, this means the process to build and use the task is

  1. Clone the repo https://github.com/rfennell/vNextBuild.git
  2. In the root of the repo use gulp to build the task
  3. Use tfx to upload the task to your TFS or VSO instance

See http://realalm.com/2015/07/31/uploading-a-custom-build-vnext-task/ and http://blog.devmatter.com/custom-build-tasks-in-vso/ for a good walkthroughs of building tasks, the process is the same for mine and Microsoft’s tasks.

IMPORTANT NOTE: This task is only for on premises TFS vNext build instances connected to either an on premises TFS or VSO. Typemock at the time of writing this post does not support VSO’s host build agents. This is because the registration of Typemock requires admin rights on the build agent which you only get if you ‘own’ the build agent VM

Once the task is installed on your TFS/VSO server you can use it in vNext builds. You will note that it takes all the same parameters as the standard VSTest task (it will usually be used as a replacement when there are Typemock Isolator based tests in a solution). The only addition to the parameters are the three parameters for Typemock licensing and deployment location.

image

Using the task allows tests that require Typemock Isolator to pass. So test that if run with the standard VSTest task give

image

With the new task gives

image

An alternative to setting a build quality on a TFS vNext build

TFS vNext builds do not have a concept of build quality unlike the old XAML based builds. This is an issue for us as we used the changing of the build quality as signal to test a build, or to mark it as released to a client (this was all managed with my TFS Alerts DSL to make sure suitable emails and build retention were used).

So how to get around this problem with vNext?

I have used Tag on builds, set using the same REST API style calls as detailed in my post on Release Management vNext templates. I also use the REST API to set the retention on the build, so I actually now don’t need to manage this via the alerts DSL.

The following script, if used to wrapper the calling of integration tests via TCM, should set the tags and retention on a build


function Get-BuildDetailsByNumber
{
    param
    (
        $tfsUri ,
        $buildNumber,
        $username,
        $password

    )

    $uri = "$($tfsUri)/_apis/build/builds?api-version=2.0&buildnumber=$buildNumber"

    $wc = New-Object System.Net.WebClient
    if ($username -eq $null)
    {
        $wc.UseDefaultCredentials = $true
    } else
    {
        $wc.Credentials = new-object System.Net.NetworkCredential($username, $password)
    }
    write-verbose "Getting ID of $buildNumber from $tfsUri "

    $jsondata = $wc.DownloadString($uri) | ConvertFrom-Json
    $jsondata.value[0]
 
}

function Set-BuildTag
{
    param
    (
        $tfsUri ,
        $buildID,
        $tag,
        $username,
        $password

    )

 
    $wc = New-Object System.Net.WebClient
    $wc.Headers["Content-Type"] = "application/json"
    if ($username -eq $null)
    {
        $wc.UseDefaultCredentials = $true
    } else
    {
        $wc.Credentials = new-object System.Net.NetworkCredential($username, $password)
    }
   
    write-verbose "Setting BuildID $buildID with Tag $tag via $tfsUri "

    $uri = "$($tfsUri)/_apis/build/builds/$($buildID)/tags/$($tag)?api-version=2.0"

    $data = @{value = $tag } | ConvertTo-Json

    $wc.UploadString($uri,"PUT", $data)
   
}

function Set-BuildRetension
{
    param
    (
        $tfsUri ,
        $buildID,
        $keepForever,
        $username,
        $password

    )

 
    $wc = New-Object System.Net.WebClient
    $wc.Headers["Content-Type"] = "application/json"
    if ($username -eq $null)
    {
        $wc.UseDefaultCredentials = $true
    } else
    {
        $wc.Credentials = new-object System.Net.NetworkCredential($username, $password)
    }
   
    write-verbose "Setting BuildID $buildID with retension set to $keepForever via $tfsUri "

    $uri = "$($tfsUri)/_apis/build/builds/$($buildID)?api-version=2.0"
    $data = @{keepForever = $keepForever} | ConvertTo-Json
    $response = $wc.UploadString($uri,"PATCH", $data)
   
}


# Output execution parameters.
$VerbosePreference ='Continue' # equiv to -verbose

$ErrorActionPreference = 'Continue' # this controls if any test failure cause the script to stop

 

$folder = Split-Path -Parent $MyInvocation.MyCommand.Definition

write-verbose "Running $folder\TcmExec.ps1"

 

& "$folder\TcmExec.ps1" -Collection $Collection -Teamproject $Teamproject -PlanId $PlanId  -SuiteId $SuiteId -ConfigId $ConfigId -BuildDirectory $PackageLocation -TestEnvironment $TestEnvironment -SettingsName $SettingsName write-verbose "TCM exited with code '$LASTEXITCODE'"
$newquality = "Test Passed"
$tag = "Deployed to Lab"
$keep = $true
if ($LASTEXITCODE -gt 0 )
{
    $newquality = "Test Failed"
    $tag = "Lab Deployed failed"
    $keep = $false
}
write-verbose "Setting build tag to '$tag' for build $BuildNumber"


$url = "$Collection/$Teamproject"
$jsondata = Get-BuildDetailsByNumber -tfsUri $url -buildNumber $BuildNumber #-username $TestUserUid -password $TestUserPwd
$buildId = $jsondata.id
write-verbose "The build $BuildNumber has ID of $buildId"
 
write-verbose "The build tag set to '$tag' and retention set to '$key'"
Set-BuildTag -tfsUri $url  -buildID $buildId -tag $tag #-username $TestUserUid -password $TestUserPwd
Set-BuildRetension -tfsUri $url  -buildID $buildId  -keepForever $keep #-username $TestUserUid -password $TestUserPwd

# now fail the stage after we have sorted the logging
if ($LASTEXITCODE -gt 0 )
{
    Write-error "Test have failed"
}

If all the tests pass we see the Tag being added and the retention being set, if they fail just a tag should be set

image

$ErrorActionPreference = 'Continue'

Cannot create an MSDeploy package for an Azure Web Job project as part of an automated build/

I like web deploy as a means to package up websites for deployment. I like the way I only need to add

/p:DeployOnBuild=True;PublishProfile=Release

as an MSBuild argument to get the package produced as part of an automated build. This opening up loads of deployment options

I recently hit an issue packaging up a solution that contained an Azure WebSite and an Azure Web Job (to be hosted in the web site). It is easy to add the web job so that it is included in the Web Deploy package. Once this was done we could deploy from Visual Studio, or package to the local file system and see the web job EXE in the app_data\jobs folder as expected.

The problems occurred when we tried to get TFS build to create the deployment package using the arguments shown above. I got the error

The value for PublishProfile is set to 'Release', expected to find the file at 'C:\vNextBuild\_work\4253ff91\BM\Src\MyWebJob\Properties\PublishProfiles\Release.pubxml' but it could not be found.

The issue is that there is a Publish target for the web jobs project type, but if run from Visual Studio it actually creates a ClickOnce package. This wizard provides no means create an MSDeploy style package.

MSBuild is getting confused as it expects there to be this MSDeploy style package definition for the web job projects, even though it won’t actually use it as the Web Job EXE will be copied into the web site deployment package.

The solution was to add a dummy PublishProfiles\Release.pubxml file into the properties folder of the web jobs project.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <WebPublishMethod>Package</WebPublishMethod>
    <LastUsedBuildConfiguration>Release</LastUsedBuildConfiguration>
    <LastUsedPlatform>Any CPU</LastUsedPlatform>
    <SiteUrlToLaunchAfterPublish />
    <LaunchSiteAfterPublish>True</LaunchSiteAfterPublish>
    <ExcludeApp_Data>False</ExcludeApp_Data>
    <DesktopBuildPackageLocation />
    <PackageAsSingleFile>true</PackageAsSingleFile>
    <DeployIisAppPath />
    <PublishDatabaseSettings/>
    </PropertyGroup>
</Project>

Note: I had to add this file to source control via the TFS Source Code Explorer as Visual Studio does not allow you add folders/files manually under the properties folder.

Once this file was added my automated build worked OK, and I got my web site package including the web job.