But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

Why have I got a ‘.NETCore50’ and a ‘netcore50’ folder in my nuget package?

I recently posted on how we were versioning our Nuget packages as part of a release pipeline. In test we noticed that the packages being produced by this process has an extra folder inside them.

image 

We expected there to be a netcore50 folder, but not a .NETCore50 folder. Strangely if we build the package locally we only saw the expect netcore50 folder. The addition of this folder did not appear to be causing any problem, but I did want to find out why it had appeared and remove it as it was not needed.

Turns out the issue was the version of Nuget.exe, the automatically installed version on the on-prem TFS build agent was 3.2, my local copy 3.4. As soon as I upgraded the build box’s nuget.exe version to 3.4 the problem went away

Experiences versioning related sets of NuGet packages within a VSTS build

Background

We are currently packaging up a set of UX libraries as NuGet packages to go on our internal NuGet server. The assemblies that make up the core of this framework are all in a single Visual Studio solution, however it makes sense to distribute them as a set of NuGet packages as you might not need all the parts in a given project. Hence we have a package structure as follows…

  • BM.UX.Common
  • BM.UX.Controls
  • BM.UX.Behaviours
  • etc…

There has been much thought on the versioning strategy of these packages. We did consider independent versioning of each of these fundamental packages, but decided it was worth the effort, keeping their versions in sync was reasonable  i.e. the packages have the same version number and are released as a set.

Now this might not be the case for future ‘extension’ packages, but it is an OK assumption for now, especially as it makes the development cycle quicker/easier. This framework is young and rapidly changing, there are often changes in a control that needs associated changes in the common assembly; it is hence good that a developers does not have to check-in a change on the common package before they can make an associated changed to the control package whist debugging a control prior to it being released.

However, this all meant it was important to make sure the package dependencies and versions are set correctly.

Builds

We are using Git for this project (though this process is just as relevant for TFVC) with a development branch and a master branch. Each branch has its own CI triggered build

  • Development branch build …
    • Builds the solution
    • Runs Unit tests
    • Does SonarQube analysis
    • DOES NOT store any built artifacts
    • [Is used to validate Pull requests]
  • Master branch build …
    • Versions the code
    • Builds the solution
    • Runs Unit tests
    • Creates the NuGet Packages
    • Stores the created packages (to be picked up by a Release pipeline for publishing to our internal NuGet server)

Versioning

So within the Master build we need to do some versioning, this needs to be done to different files to make sure the assemblies and the NuGet packages are ‘stamped’ with the build version.

We get this version for the build number variable, $(Build.BuildNumber), we use the format $(Major).$(Minor).$(Year:yy)$(DayOfYear).$(rev:r)  e.g. 1.2.16123.3

Where

  • $(Major) and $(Minor) build variables we manage (actually our release pipeline updates the $(Minor) on every successful release to production using a VSTS task)
  • $(Year:yy)$(DayOfYear) gives a date in the form 16123
  • and $(rev:r) is a count of builds on a given day

We have chosen to use this number format to version both the assemblies and Nuget packages, if you have different plans, such as semantic versioning , you will need to modify this process a bit.

Assemblies

The assemblies themselves are easy to version, we just need to set the correct value in their assemblyinfo.cs or assemblyinfo.vb files. I used my Assembly versioning VSTS task to do this

NuGet Packages

The packages turn out to be a bit more complex. Using the standard NuGet Packager task there is a checkbox to say to use the build number as the version. This works just fine versioning the actual package, adding the –Version flag to the package command to override the value in the project .nuspec file. However it does not help with managing the versions of any dependant packages in the solution, and here is why. In our build …

  1. AssemblyInfo files updated
  2. The solution is built, so we have version stamped DLLs
  3. We package the first ‘common’ Nuget package (which has no dependencies on other projects in the solution) and it is versioned using the –version setting, not the value in it’s nuspec file.
  4. We package the ‘next’ Nuget package, the package picks up the version from the –version flag (as needed), but it also needs to add a dependency to a specific version of the ‘common’ package. We pass the –IncludeReferencedProjects  argument to make sure this occurs. However, Nuget.exe gets this version number from  the ‘common’ packages .nuspec file NOT the package actually built in the previous step. So we end up with a mismatch.

The bottom line is we need to manage the version number in the .nuspec file of each package. So more custom VSTS extensions are needed.

Initially I reused my Update XML file task, passing in some XPath to select the node to update, and this is a very valid approach if using semantic versioning as it is a very flexible way yo build the version number. However, in the end I added an extra task to my versioning VSTS extension for Nuget to make my build neater and consistent with my other versions steps.

Once all the versioning was done I could create the packages. I ended up with a build process as shown below

image

A few notes about the NuGet packaging

  • Each project I wish to create a Nuget package for has a nuspec file of the same ‘root’ name in the same folder as the csproj eg. mypackage.csproj and mypackage.nuspec. This file contains all descriptions, copyright details etc.
  • I am building each package explicitly, I could use wildcards in the ‘Path/Pattern to nuspec files’ property, I choose not to at this time. This is down to the fact I don’t want to build all the solution’s package at this point in time.
  • IMPORTANT I am passing in the .csproj file names, not the .nuspec file names to the ‘Path/Pattern to nuspec files’ property. I found I had to do this else the   –IncludeReferencedProjects  was ignored. The Nuget documentation seems to suggest as long as the .csproj and .nuspec files have the same ‘root’ name then you could reference the .nuspec file but this was not my experience
  • I still set the flag to use the build version to version the package – this is not actually needed as the .nuspec file has already been update
  • I pass in the  –IncludeReferencedProjects  argument via the advanced parameters, to pick up the project dependancies.

Summary

So now I have a reliable way to make sure my NuGet packages have consistent version numbers 

Tidy up those VSTS release pipelines with meta-tasks

Do you have repeating blocks in your VSTS release pipelines?

I certainly do. A common one is to run a set of functional test, so I need to repeatedly …

  1. Deploy some test files to a VM
  2. Deploy a test agent to the VM – IMPORTANT I had not realised you can only run one test run against this deployed agent. You need to redeploy it for the next run
  3. Run my tests
  4. … and repeat for next test type/configuration/test plan/DLL etc.

 

In the past this lead to a lot of repeat tasks in my release pipeline, all very messy.

Now in VSTS we have the option of  Meta-tasks, these allow tasks to be grouped into in-effect functions with their own properties.

 

image

In the above screen shot below you can see I use a meta-task ‘Run Tests’ that wrappers the four tasks shown below.

image

Much neater, but as you might expect with something new I have come across a few minor gotchas

  • You cannot order the list of properties for the meta-task
  • This is a problem as the first one is used to generate the instance name in the pipeline. No a major problem you can always edit it.
  • Meta-tasks properties are auto-detected from any variables used with in the meta-task tasks, the auto-detection mechanism is case sensitive, unless the rest of VSTS variable handling. So be careful to not end up with duplicates.

That all said, I think this is big step forward in readability and reuse for release management

New version of my generate release notes task–now with authentication options

I have just released 1.4.7 of the release notes VSTS extension. This provides a new advanced options that allows you to switch the authentication model.

The default remains the same i.e. use a personal access token provided by the server, but you have the option to enable use of the 'defaultcredentials' (via the advanced properties). If this is done the account the build agent is running as is used. Hopefully this should fix the 401 issues some people have been seeing when using the task with on-prem TFS.

For most people the default PAT model should be fine

Windows 10 Anniversary (Build 1607) messed up my virtual NAT Switch – a fix

I use a virtual NAT Switch to allow my VMs to talk to the outside world. The way I do this is documented in this post, based on the work of  Thomas Maurer. The upgrade to Windows 10 Anniversary messed this up, just seemed to loose the virtual network completely, VMs failed to start with invalid configurations and would not even start.

I had to recreate my NATSwitch using Thomas’s revised instructions, but I did have an problem. The final ‘New-NetNat’ command failed with  a ‘The parameter is incorrect.’ error. I think the issue was that there was debris left from the old setup (seems Microsoft removed the NatSwitch interface type). I could find no way to remove the old NATSwitch as it did not appear in the list in PowerShell and there is no UI remove option. So I just ended up disabling it via the UI and this seemed to do the trick

image

My VMs seem happy again talking to the outside world

Out with the Band in with the Garmin

I have been using the Microsoft Band (both version Band1 and Band2) since they came out, and been reasonably happy. However, a year or so on my issues with it have remained the same

  • Poor battery life, I can live with charging it each day, but even with GPS Power-saver mode on I can’t go for any exercise over about 4 hours (bit of an issue for longer bike rides)
  • It is not waterproof, so no swimming (and worried doing the washing up)

Also there seem to be some build issues with the robustness of the Band2. I had to get mine replaced due to it not accepting recharging and the forums seems to report people suffering problems with the wrist strap splitting. That said, the warrantee service seems excellent, no complaints there, mine was swapped without any issue in a couple of days

In the end however, I decided it was time to to check out alternatives and picked the Garmin Vivoactive HR; basically the Garmin equivalent to the Band in feature set and price (it is a little more expensive in the UK)

 

image

 

I have to say a couple of weeks in I am very pleased. It fixes those two major issues for me. Most importantly I seem to need charging it only about every 5 days or so, that is with with an hour or two of full activity tracking each day. The specs claim 10 hour+ for full activity tracking on a charge. Also it is waterproof and allows activity tracking for pool based swimming (swim mode is lap based and has no GPS enabled so less use for open water).

That all said there are still issues

  • The Bluetooth link to my Windows Phone 10 is a little temperamental for things like notifications and sync -  a restart usually fixes everything (but hey it fully supports Windows Phone 10 not just Android and iPhone!)
  • Shame they disable heart rate monitor for swimming (signal not reliable enough, unless you pair with a chest strap it seems)
  • Lack of open water swimming tracking (see above – but of you want full multisport tracking look at the Garmin 920XT, their top of the range watch it does it all)

But I think these are all minor issues for me, and the third party apps store for the device help such as adding triathlon support which attempts HR monitoring for swimming, without needing to upgrade to the 920XT.

So a good alternative to theBand2?

For me yes, it addresses my key issues. Band2 is a good fitness tracker with unique styling, but if swimming or longer activities are your thing I think the Garmin Vivoactive HR has it.

New Build Management VSTS tasks

Just published a new VSTS extension with a couple of tasks in it. The aim to to help formalise the end of a release process. The tasks

  • Allow you to set the retension ‘keep forever’ flag on a build (or all builds linked to a release)
  • Update increment a build variable e.g. all or part of a version number, in a build (or all builds linked to a release)

The first just replicates functionality I used to have in house for builds

The second one is important to me as once I have released to production a version of a product I never want to generate another build with the same base version number. For example we version stamp all our DLLs/builds with a version number in form

$(Major).$(Minor).$(year:yy)$(dayofyear).(rev:r)     e.g. 1.2.16170.2

Where the $(Major) and $(Minor) are build variables we set manually (we decide when we increment a major or minor release) and the second two blocks guarantee a unique build number every time. It is too easy to forget to manually increment the Major or Minor build variable during a release. This task means I don’t need to remember to set the value of one or both of these. I can either set an explicit value or just get it to auto-increment. I usually auto increment the Minor value as a default, doing a manual reset of both the Major and Minor if it is a major release.

NOTE: You do have to add some permissions to the build service account else this second task fails with a 403 permission error – so read the WIKI

Life gets better in Visual Studio Code for PowerShell

I have been using Visual Studio Code for PowerShell development, but got a bit behind on reading release notes. Today I just realised I can make my Integrated Terminal a Code a PowerShell instance.

In File > Preferences > user Settings (settings.json) enter the following

 

// Place your settings in this file to overwrite the default settings
{
     // The path of the shell that the terminal uses on Windows.
    "terminal.integrated.shell.windows": "C:\\windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe"
}

Now my terminal is a PowerShell instance, and you can see it has loaded by profile so POSH Git is work as well

 

image

 

So I think we have reached the goodbye PowerShell ISE point

Gotcha’s when developing VSTS Build Extension

I recently posted on my development process for VSTS Extensions, it has been specifically PowerShell based build ones I have been working on. During this development I have come across a few more gotcha’s that I think are worth mentioning

32/64 bit

The VSTS build agent launches PowerShell 64bit (as does the PowerShell command line on dev PC), but VSCode launches it 32bit. Whilst working my StyleCop extension this caused me a problem as StyleCop it seems can only load dictionaries for spell checking based rules when in a 32bit shell. So my Pester tests for the extension worked in VSCode but failed at the command line and within a VSTS build

After many hours my eventual solution was to put some guard code in my scripts to force a reload in 32bit mode

param
(
    [string]$treatStyleCopViolationsErrorsAsWarnings,
    [string]$maximumViolationCount,
    … other params
)

if ($env:Processor_Architecture -ne "x86")  
{
    # Get the command parameters
    $args = $myinvocation.BoundParameters.GetEnumerator() | ForEach-Object {$($_.Value)}
    write-warning 'Launching x86 PowerShell'
    &"$env:windir\syswow64\windowspowershell\v1.0\powershell.exe" -noprofile -executionpolicy bypass -file $myinvocation.Mycommand.path $args
    exit
}
write-verbose "Running in $($env:Processor_Architecture) PowerShell"

... rest of my code

 

The downside of this trick is that you can’t pass return values back as you swapped execution process. For the type of things I am doing with VSTS tasks this not an issue as the important data has usually be dropped to a file which is accessible by everything, such as test results.

For a worked sample of production code and Pester tests see by GitHub repo.

Using Modules

In the last post I mentioned the problem when trying to run Pester tests against scripts, the script content is executed. I stupidly did not mention the obvious solution of moving all the code into functions in a PowerShell modules. This makes it easier to write tests for all bar the outer wrapper .PS1 script that is called by the VSTS agent.

Again see by GitHub repo so a good sample. Note how I have split out the files so that I have

  • A module that contains the functions I can test via Pester
  • A .PS1 script called by VSTS (this will run 64bit) where I deal with interaction with VSTS/TFS
  • An inner PS1 string that we force into 32bit mode as needed (see above)

Hacking around on your code

You always get to the point I find when developing things like VSTS build tasks that you want to make some quick change to try something without the full development/build/release cycle. This is in effect the local development stage, it is just build task development makes with awkward. It is hard to fully test a task locally, it need to be deployed within a build

I have found a way to help here is to use a local build agent, you can then get at the deployed task and edit the .PS1 code. The important bit to node is that the task will not be redeployed so you local ‘hack’ can be tested within a real TFS build without having to increment the task’s version and redeploy.

Hacky but handy to know.

You of course do need to make sure you hacked code is eventually put through your formal release process.

And maybe something or nothings…

I may have seen these issues, but have not got to the bottom of them, so they may not be real issues

  • The order parameters are declared in a task.json file seems to need to match the order they are declared in the .PS1 file call. I had thought they we associated by name not order, but in one task they all got transposed until I fixed the order.
  • The F5 dev debug cycle is still a little awkward with VSCode, sometime it seems to leave stuff running and you get high CPU utilisation – just restart the VSCode  - the old fix!
  • If using the 32 bit relaunch discussed above write-verbose messages don’t awlays seem to show up in the VSTS log, I assume a –verbose parameter is being lost somewhere, or it is the spawning of another PowerShell instance that cause the problem.

SO again I hope these tips help with your VSTS extension development