As I posted recently I have been trying to add more functional tests to the VSTS based release CI/CD pipeline for my VSTS Extensions, and as I noted depending on how you want to run your tests e.g. trigger sub-builds, you can end up with scheduling deadlocks where a single build agent is scheduling the release and trying to run a new build. The answer is to use a second build agent in a different agent pool e.g. if the release is running on the Hosted build agent use a private build agent for the sub-build, or of course just pay for more hosted build instances.
The problem with a private build agent is where to run it. As my extensions are a personal project I don’t have a corporate Hyper-V server to run any extra private agents on, as I would have for an company projects. My MVP MSDN Azure benefits are the obvious answer, but I want any agents to be cheap to run, so I don’t burn through all my MSDN credits for a single build agent.
To this end I created a Windows Server 2016 VM in DevLabs (I prefer to create my VMs in DevLabs as it makes it easier tidying up of my Azure account) using an A0 sizing VM. This is tiny so cheap; I don’t intend to ever do a build on this agent, just schedule releases, so need to install few if any tools, so the size should not be an issue. To further reduce costs I used the auto start and stop features on the VM so it is only running during the hours I might be working. So I get an admittedly slow and limited private build agent but for less that $10 a month.
As the VM is small it makes sense to not run a GUI. This means when you RDP to the new VM you just get a command prompt. So how do you get the agent onto the VM and setup? You can’t just open a browser to VSTS or cut and paste a file via RDP, and I wanted to avoid the complexity of having to open up PowerShell remoting on the VM.
The process I used was as follows:
- In VSTS I created a new Agent Pool for my Azure hosted build agents
- In the Azure portal, DevLabs I created a new Windows Server 2016 (1709) VM
- I then RDP’d to my new Azure VM, in the open Command Prompt I ran PowerShell
- As I was in my users home directory, I cd’d into the downloads folder
- I then ran the following PowerShell command to download the agent (you can get the current URI for the agent from your VSTS Agent Pool ‘Download Agent’ feature, but an old version will do as it will auto update.
invoke-webrequest -UseBasicParsing -uri https://github.com/Microsoft/vsts-agent/releases/download/v2.124.0/vsts-agent-win7-x64-2.124.0.zip -OutFile vsts-agent-win7-x64-2.124.0.zip
- You can then follow the standard agent setup instructions from the VSTS Agent Pool ‘Download Agent’ feature
mkdir agent ; cd agent
Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory(“$HOMEDownloadsvsts-agent-win7-x64-2.124.0.zip”, “$PWD”)
- I then configured the agent to run as a service, I exited back to the command prompt to do this this, so the commands were
I now had an other build agent pool to use in my CI/CD pipelines at a reasonable cost, and the performance was not too bad either.
As time passes I have found there is a need for more cross platform VSTS extensions as there is more uptake of VSTS beyond it’s historic Microsoft platform based roots.
Historically most of my extensions have been Powershell based. Now this is not a fundamental problem for cross platform usage. this is due to the availability of Powershell Core. However, Typescript and Node.JS is a better fit I think in many cases. This has caused me to revise the way I structure my repo and build my VSTS extensions to provide a consistent understandable process. My old Gulp based process for Typescript was too complex and inconsistent between tasks, it even confused me! My process revisions have been documented in my vNextBuild Github’s WIKI, so I don’t propose too repeat the bulk of the content here.
That said, it is worth touching on why I did not use YO VSTS. If I were starting this project again I would certainly look at this tool to build out my basic repo structure. Also I think I would look at a separate repo per VSTS extension, as opposed to putting them all in one repo. However, this project pre-dates the availability of YO VSTS , hence the structure I have is different. Also as people have forked the repo I don’t intend to introduce breaking changes by splitting the repo. That all said my Typescript/Node.JS process is heavily influenced by the structures and NPM script used in YO VSTS, so people should find the core processes familiar i.e.
- At the command prompt
- CD into an extension folder you wish to build (this contains the package.json file)
- Get all the NPM packages npm install as defined in the package.json
- Run TSLink and transpile the TypeScript to Node.JS. There is a single command npm run build to do this, but they can be rerun individually using npm run lint and npm run transpile
- Run an unit tests you have npm run test-no-logging (Note the npm run test script is used by the CI/CD process to dump the test results for uploading to VSTS logging. I tend to find when working locally that just dumping the test results to the console is enough.
- Package the the resultant .JS files npm run package. This script does two jobs, it remove any NPM modules that are set as -savedev i.e. only used in development and not production and also copies the required files to the correct locations to be packaged into a VSIX file. NOTE after this command is run you need to rerun npm install to reinstall the development NPM packages prior to be able to run tests etc. locally
This is complement by an enhance VSTS CI/CD process that has a far greater focus on automated testing as well as packaging the VSIX files and releasing to the VSTS marketplace
Check the vNextBuild Github’s WIKI for more details
Updated 22 Mar 2016 This task is available in the VSTS Marketplace
If you are using Pester to unit test your PowerShell code then there is a good chance you will want to include it in your automated build process. To do this, you need to get Pester installed on your build machine. The usual options would be
If you own the build agent VM then any of these options are good, you can even write the NuGet restore into your build process itself. However there is a problem, both the first two options need administrative access as they put the Pester module in the $PSModules folder (under ‘Program Files’); so these can’t be used on VSTS’s hosted build system, where your are not an administrator
So this means you are left with copying the module (and associated functions folder) to some local working folder and running it manually; but do you really want to have to store the Pester module in your source repo?
My solution was to write a vNext build tasks to deploy the Pester files and run the Pester tests.
The task takes two parameters
- The root folder to look for test scripts with the naming convention *.tests.ps1. Defaults to $(Build.SourcesDirectory)*
- The results file name, defaults to $(Build.SourcesDirectory)Test-Pester.XML
The Pester task does not in itself upload the test results, it just throws and error if tests fails. It relies on the standard test results upload task. Add this task and set
- it to look for nUnit format files
- it already defaults to the correct file name pattern.
- IMPORTANT: As the Pester task will stop the build on an error you need to set the ‘Always run’ to make sure the results are published.
Once all this is added to your build you can see your Pester test results in the build summary
You can find the task in my vNextBuild repo
Updated 22 Mar 2016 This tasks is available in the VSTS Marketplace
I have previously posted on how a PowerShell script can be used to run StyleCop as part of vNext VSTS/TFS build. Now I have more experience with vNext tasks it seemed a good time to convert this PowerShell script into a true task that can deploy StyleCop and making it far easier to expose the various parameters StyleCop allows.
To this end I have written a new StyleCop task that can be found in my vNext Build Repo, this has been built to use the 18.104.22.168 release of StyleCop (so you don’t need to install StyleCop in the build machine, so it works well on VSTS).
To use this task:
- Clone the repo
- Build the tasks using Gulp
- Upload the task you require to your VSTS or TFS instance
Once this is done you can add the task to your build. You probably won’t need to set any parameters as long as you have settings.stylecop files to define your StyleCop ruleset in the same folders as your .CSPROJ files (or are happy default rulesets).
If you do want to set parameters your options are:
- TreatStyleCopViolationsErrorsAsWarnings – Treat StyleCop violations errors as warnings, if set to False any StyleCop violations will cause the build to fail (default false).
And on the advanced panel
- MaximumViolationCount – Maximum violations before analysis stops (default 1000)
- ShowOutput – Sets the flag so StyleCop scanner outputs progress to the console (default false)
- CacheResults – Cache analysis results for reuse (default false)
- ForceFullAnalysis – Force complete re-analysis (default true)
- AdditionalAddInPath – Path to any custom rule sets folder, the directory cannot be a sub directory of current directory at runtime as this is automatically scanned. This folder must contain your custom DLL and the Stylecop.dll and Stylecop.csharp.cs else you will get load errors
- SettingsFile – Path to single settings files to use for all analysis (as opposed to settings.stylecop files in project folders)
When you run the build with the new task you should expect to see a summary of the StyleCop run on the right