How do handle PRs for Azure DevOps YAML Pipelines if the YAML templates are in a different repo?

Azure DevOps YAML base pipelines allow the pipeline definitions to be treated like any other code. So you make changes in a branch and PR them into the main/trunk when they are approved.

This works well if all the YAML files are in the same repo, but not so well if you are using YAML templates and the templated YAML is stored in a different repo. This is because an Azure DevOps PR is limited to a single repo. So testing a change to a YAML template in a different repo needs a bit of thought.

Say for example you have a template called core.yml in a repo called YAMLTemplates and you make a change to it and start a PR. Unless you have a test YAML pipeline in that repo, which is not a stupid idea, but not always possible depending on the complexity of your build process, there is no way to test the change inside that repo.

The answer is to create a temporary branch in a repo that consumes the shared YAML template. In this temporary branch make an edit to the repository setting that references the shared YAML repo to point to the update branch contain the PR

resources: 
repositories:
  - repository: YAMLTemplates
   type: git
   name: 'Git Project/YAMLTemplates'
# defaults to ref: 'refs/heads/master'
ref: 'refs/heads/newbranch'

You don’t need to make any change to the line where the template is used

extends:  
template: core.yml@YAMLTemplates
  parameters:
    customer: ${{parameters.Customer}}
    useSonarQube: ${{parameters.useSonarQube}}

You can then use this updated pipeline to validated your PR. Once you are happy it works you can

  1. Complete the PR in the YAML Templates repo
  2. Delete the temporary branch in your consuming repo.

How can I automatically create Azure DevOps Release Notes and how can I publish them?

A question I am often asked when consulting on Azure DevOps is ‘how can I automatically create release notes and how can I publish them?’.

Well it is for just this requirement that I have written a set of Azure DevOps Pipeline Tasks

  • Release Note Generator – to generate release notes. I strongly recommend this Cross-platform Node-based version. I plan to deprecate my older PowerShell version in the not too distant future as it uses ‘homegrown logic’, as opposed to standard Azure DevOps API calls, to get associated items.
  • Wiki Updater – to upload a page tot a WIKI.
  • WIKI PDF Generator – to convert a generated page, or whole WIKI, to PDF format.

So lets deal with these tools in turn

Generating Release Notes

The Release Note task generates release notes by getting the items associated with a build (or release) from the Azure DevOps API and generating a document based on a Handlebars based template.

  • The artefacts that can be included in the release notes are details of the build/release and associated Work Items, Commits/Changesets, Tests and Pull Requests.
  • Most of the sample templates provided are for markdown format files. However, they could easily be converted for other text-based formats such as HTML if needed.
  • The use of Handlebars are the templating language makes for a very flexible and easily extensible means of document generation. There are sample of custom extensions provided with the templates

Sample YAML for this task is as follows, not it is using an inline template but it is possible to also load the template from a file path

        - task: richardfennellBM.BM-VSTS-XplatGenerateReleaseNotes.XplatGenerate-Release-Notes.XplatGenerateReleaseNotes@3
          displayName: 'Generate Release Notes'
          inputs:
            outputfile: '$(System.DefaultWorkingDirectory)inline.md'
            outputVariableName: OutputText
            templateLocation: InLine
            inlinetemplate: |
              # Notes for build 
              **Build Number**: {{buildDetails.id}}
              **Build Trigger PR Number**: {{lookup buildDetails.triggerInfo 'pr.number'}} 

              # Associated Pull Requests ({{pullRequests.length}})
              {{#forEach pullRequests}}
              {{#if isFirst}}### Associated Pull Requests (only shown if  PR) {{/if}}
              *  **PR {{this.id}}**  {{this.title}}
              {{/forEach}}

              # Builds with associated WI/CS ({{builds.length}})
              {{#forEach builds}}
              {{#if isFirst}}## Builds {{/if}}
              ##  Build {{this.build.buildNumber}}
              {{#forEach this.commits}}
              {{#if isFirst}}### Commits {{/if}}
              - CS {{this.id}}
              {{/forEach}}
              {{#forEach this.workitems}}
              {{#if isFirst}}### Workitems {{/if}}
              - WI {{this.id}}
              {{/forEach}} 
              {{/forEach}}

              # Global list of WI ({{workItems.length}})
              {{#forEach workItems}}
              {{#if isFirst}}## Associated Work Items (only shown if  WI) {{/if}}
              *  **{{this.id}}**  {{lookup this.fields 'System.Title'}}
                - **WIT** {{lookup this.fields 'System.WorkItemType'}} 
                - **Tags** {{lookup this.fields 'System.Tags'}}
              {{/forEach}}

              {{#forEach commits}}
              {{#if isFirst}}### Associated commits{{/if}}
              * ** ID{{this.id}}** 
                -  **Message:** {{this.message}}
                -  **Commited by:** {{this.author.displayName}} 
                -  **FileCount:** {{this.changes.length}} 
              {{#forEach this.changes}}
                    -  **File path (TFVC or TfsGit):** {{this.item.path}}  
                    -  **File filename (GitHub):** {{this.filename}}  
              {{/forEach}}
              {{/forEach}}

How to Publish The Notes

Once the document has been generated there is a need for a decision as to how to publish it. TThere are a few options

  • Attach the markdown file as an artefact to the Build or Pipeline. Note you can’t do this with a UI based Releases as they have no concept of artefacts, but this is becoming less of a concern as people move to multistage YAML.
  • Save in some other location e.g Azure Storage or if on-premises a UNC file share
  • Send the document as an email – I have used Rene van Osnabrugge Send Email Task for this job.
  • Upload it to a WIKI using my WIKI Updater Task
  • Convert the markdown release note document, or the whole WIKI, to a PDF and use any of the above options using first my WIKI PDF Exporter Task then another task.

I personally favour the 1st and 4th options used together. Attachment to the pipeline and then upload the document to a WIKI

A sample of suitable YAML is shown below, uploading the document to an Azure DevOps WIKI. Please note that the repo URL and authentication can trip you up here so have a good read of the provided documentation before you use this task.

  - task: richardfennellBM.BM-VSTS-WIKIUpdater-Tasks.WikiUpdaterTask.WikiUpdaterTask@1
          displayName: 'Git based WIKI Updater'
          inputs:
            repo: 'dev.azure.com/richardfennell/Git%20project/_git/Git-project.wiki'
            filename: 'xPlatReleaseNotes/build-Windows-handlebars.md'
            dataIsFile: true
            sourceFile: '$(System.DefaultWorkingDirectory)inline.md'
            message: 'Update from Build'
            gitname: builduser
            gitemail: 'build@demo'
            useAgentToken: true

But when do I generate the release notes?

I would suggest you always generate release notes every build/pipeline i.e. a document of the changes since the last successful build/pipeline of that build definition. This should be attached as an artefact.

However, this per build document will usually too granular for use as ‘true’ release notes i.e. something to hand to a QA team, auditor or client.

To address this second use case I suggest, within a multistage YAML pipeline (or a UI based release), having a stage specifically for generating release notes.

My task has a feature that it will check for the last successful release of a pipeline/release to the stage it is defined in, so will base the release note on the last successful release to that given stage. If this ‘documentation’ stage is only run when you are doing a ‘formal’ release, the release note generated will be since the last formal release. Exactly what a QA team or auditor or client might want.

In conclusion

So I hope that this post provides some ideas as to how you can use my tasks generate some useful release notes.

Fix For: ‘The pipeline is not valid error: Unable to resolve latest version’ on an Azure DevOps YAML pipeline

The Issue

I have an Azure DevOps multi-stage YAML pipeline that started giving the error `The pipeline is not valid error: Unable to resolve latest version for pipeline templates: this could be due to inaccessible pipeline or no version is available` and failing instantly.

image

The Solution

This is not the most helpful message, but after some digging I found the problem.

The pipeline used another pipeline as a resources

resources: 
pipelines:
- pipeline: templates
source: QueuesAndFunctionsDemo-CI
branch: master

This referenced build had failed, so there was no successful build resources to load, hence the error.

Once the problem with this reference build was fixed the error message went away and I could trigger my build

Exporting Release Notes and WIKIs as PDFs using a new Azure DevOps Extension that wrappers AzureDevOps.WikiPDFExport

A common question I get when people are using my Release Notes task for Azure DevOps is whether it is possible to get the release notes as a PDF.

In the past, the answer was that I did not know of any easy way. However, I have recently come across a command line tool by Max Melcher called AzureDevOps.WikiPDFExport that allows you to export a whole WIKI (or a single file) as a PDF. Its basic usage is

  • Clone a WIKI Repo
  • Run the command line tool passing in a path to the root of the cloned repo
  • The .order file is read
  • A PDF is generated

This is a nice and simple process, but it would be nice to be able to automate this process as part of a build pipeline.

After a bit of thought, I realised I had much of the code I needed to automated the process in my WIKIUpdater extension as these tasks are based around cloning repos.

So I am please to say I have just released a new Azure DevOps extension WikiPDFExport that wrappers Max’s command line tool. It does the following

  • Downloads the latest release of the WikiPDFExport tool from GitHub to the build agent (the exe is too big to include in the VSIX package)
  • Optionally clone a Git based WIKI repo. As with my WikIUpdater tasks, you can pass credentials for Azure DevOps or GitHub
  • Generate a PDF of a single file or the whole of a Wiki folder structure (based on the .order file) that was either cloned or was already present on the agent

A sample of the YAML usage of the task is as shown below. For full documentation see the extensions wiki pages for general usage and troubleshooting and the full YAML specification

- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1 
  displayName: 'Export Single File generated by the release notes task'
  inputs:
    cloneRepo: false
    localpath: '$(System.DefaultWorkingDirectory)'
    singleFile: 'inline.md'
    outputFile: '$(Build.ArtifactStagingDirectory)PDFsingleFile.pdf'
- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1
   displayName: 'Export a public GitHub WIKI'
   inputs:
     cloneRepo: true
     repo: 'https://github.com/rfennell/AzurePipelines.wiki.git' 
     useAgentToken: false
     localpath: '$(System.DefaultWorkingDirectory)GitHubRepo' 
     outputFile: '$(Build.ArtifactStagingDirectory)PDFpublicGitHub.pdf'
- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1
   displayName: 'Export a private Azure DevOps WIKI'
   inputs:
     cloneRepo: true
     repo: 'https://dev.azure.com/richardfennell/GitHub/_git/GitHub.wiki' 
     useAgentToken: true
     localpath: '$(System.DefaultWorkingDirectory)AzRepo' 
     outputFile: '$(Build.ArtifactStagingDirectory)PDFAzrepo.pdf'

So hopefully this new extension will give teams another way to present their release notes, whether it be an export of a whole WIKI or just a single page.

Using the Post Build Cleanup Task from the Marketplace in YAML based Azure DevOps Pipelines

Disks filling up on our private Azure DevOps agents is a constant battle. We have maintenance jobs setup on the agent pools, to clean out old build working folders nightly, but these don’t run often enough. We need a clean out more than once a day due to the number and size of our builds.

To address this, with UI based builds, we successfully used the Post Build Cleanup Extension. However since we have moved many of our builds to YAML we found it not working so well. Turned out the problem was due to the way got source code.

The Post Build Cleanup task is intelligent, it does not just delete folders on demand. It check to see what the Get Source ‘Clean’ setting was when the repo was cloned and bases what it deletes on this value e.g. nothing, source, or everything. This behaviour is not that obvious.

In a UI based builds it is easy to check this setting. You are always in the UI when editing the build. However, in YAML it is easy to forget the setting, as it is one of those few values that cannot be set in YAML.

To make the post build cleanup task actually delete folders in a YAML pipeline you need to

  1. Edit the pipeline
  2. Click the ellipse menu top right
  3. Pick Triggers
  4. Pick YAML and select the ‘Get Source’ block
  5. Make sure the ‘Clean’ setting is set to ‘true’ and the right set of items to delete are selected – if this is not done the post clean up task does nothingimage
  6. You can then add the post build cleanup task the end of the steps
steps:
  - script: echo This where you do stuff
  - task: mspremier.PostBuildCleanup.PostBuildCleanup-task.PostBuildCleanup@3
    displayName: 'Clean Agent Directories'
    condition: always()

Once this is done it behaves as expected

Zwift and the joys of home networking

During the Covid 19 lock down I have been doing plenty of Zwift‘ing. However, I have started having problems getting the Zwift Companion App working reliably, when it used to work.

Basically, Zwift itself was fine, though very slow to save when exiting, but the companion app could not seem to detect that I was actively Zwift’ing, but it’s other functions were OK.

After much fiddling I found the issue was the network connection from my PC up to Zwift and nothing to do with the phone app. But in case it is of any use to others here are the steps I took to

  • Ran a WiFi network analysis app and realised that
    • My local wireless environment is now very congested, I assume as more people are working from home.
    • Both the 2.4GHz and 5Ghz network were on the same channels as other strong signals.
    • Also they were using the same SSID, which is meant to provide seamless swap-over between 2.4 and 5Ghz. But, in reality this meant there were connection problems as a connection flipped between frequencies.

This explained other problems I had seem

  • The Microsoft Direct Access VPN I use to connect to the office failing intermittently. Obviously, any problems I have connecting to the office to do work is far less important than Zwift connection issues.
  • My Samsung phone would drop calls for no reason. I now think this was when it had decided to use Wifi calling and got confused over networks.
    Note: I had fixed this by switching off Wifi calling.

To address the problems I changed the SSIDs so that my 2.4 and 5Ghz networks had different names, so that I know which one I was using. Also I moved the channels to ones not used by my neighbours

TestResult
Put the phone and the PC on the 2.4Ghz networkNo improvement, app did not work and PC slow to save
Put the phone and the PC on 5GhzSmall improvement, app still did not work but at least tried to show the in game view before it dropped out. The PC was still slow to save
Put the phone on either Wifi network but the the PC on Ethernet over Power using TPLink adaptorsThis fixed it

So it seems the problem was upload speed from my PC all along. Strange as I would have expected the 5Ghz network to be fine, even if the 2.4Ghz was not. The 5Ghz Wifi seems to perform OK on a speed test.

Anyway it is working now, but maybe it is time to consider a proper mesh network?

Bringing Stage based release notes in Multi-Stage YAML to my Cross Platform Release Notes Exension

I have just released Version 3.1.7 of my Azure DevOps Pipeline XplatGenerateReleaseNotes Extension.

This new version allows you to build release notes within a Multi-Stage YAML build since the last successful release to the current (or named) stage in the pipeline as opposed to just last fully successful build.

This gives more feature parity with the older UI based Releases functionality.

To enable this new feature you need to set the checkStage: true flag and potentially the overrideStageName: AnotherStage if you wish the comparison to compare against a stage other than the current one.

- task: XplatGenerateReleaseNotes@3
  inputs:
    outputfile: '$(Build.ArtifactStagingDirectory)releasenotes.md'
    outputVariableName: 'outputvar'
    templateLocation: 'InLine'
    checkStage: true
    inlinetemplate: |
      # Notes for build 
      **Build Number**: {{buildDetails.id}}
      ...

Timeout Errors ‘Extracting Schema’ when running SQLPackage for a Migration to Azure DevOps Services

The Problem

Whilst doing a migration from an on-premised TFS to Azure DevOps Services for a client I had a strange issue with SQLPackage.exe.

I had previously completed the dry run of the migration without any issues and started the live migration with a fully defined process and timings for a each stage.

When I came to export the detached Team Project Collection DB I ran the same command as I had for the dry run

& "C:Program FilesMicrosoft SQL Server150DACbinSqlPackage.exe" /sourceconnectionstring:”Data Source=localhostSQLExpress;Initial Catalog=Tfs_DefaultCollection;Integrated Security=True” /targetFile:C:tempTfs_DefaultCollection.dacpac /action:extract /p:ExtractAllTableData=true /p:IgnoreUserLoginMappings=true /p:IgnorePermissions=true /p:Storage=Memory 

I had expected this to take around 30 minutes. However it failed after 10 minutes with an error when trying to export the schema ‘Timeout, cannot reconnect to the database’.

This was strange as nothing had changed on the system since the dry-run. I tried all of the following with no effect

  • Just running the command again, you can hope!
  • Restarted SQL and ran the command again
  • Tried the export from SQL Management Studio as opposed to the command line , this just seems to hang at the same point.

The Solution

What resolved the problem was a complete reboot of the virtual machine. I assume the issue was some locked resource, but not idea why.


Updated 29th July 2020

I had the same problem with another client upgrade. This time a reboot did not fix it.

The solution at this site was to upgrade SQLPackage from the 140 32bit version to the 64bit 150 version. Once this was done the command ran without a problem

Getting started with Aggregator CLI for Azure DevOps Work Item Roll-up

Back in the day I wrote a tool, TFS Alerts DSL, to do Work Item roll-up for TFS. Overtime I updated this to support VSTS (as Azure DevOps was then called), it’s final version is still available in the Azure DevOps Marketplace as the Azure DevOps Service Hooks DSL. So when I recently had a need for Work Item roll-up I did consider using my own tool, just for a short while. However, I quickly realised a much better option was to use the Aggregator CLI. This is a successor to the TFS Aggregator Plug-in and is a far more mature project than my tool and actively under development.

However, I have found the Aggregator CLI a little hard to get started with. The best ‘getting started’ documentation seems to be in the command examples, but I is not that easy to find. So I thought this blog post was a good idea, so I don’t forget the details in the future.

Architecture

In this latest version of the Aggregator the functionality is delivered using Azure Functions, one per rule. These are linked to Azure DevOps Service hook events. The command line tool setup process configures all of the parts required setting up Azure resources, Azure DevOps events and managing rules.

Preparation

    1. Open the  Azure Portal
    2. Select the Azure Active Directory (AAD) instance to create an App Registration in.
    3. Create a new App Registration
      1. Create a new app registration
      2. Provide name, you can leave the rest as defaults
      3. Press Register
    4. From the root of the Azure Portal pick the Subscription you wish to create the Azure Functions in
    5. In the Access (IAM ) section grant the ‘contributor role’ for the subscription to the newly created App Registration.

Using the Aggregator CLI

At a command prompt we need to now start to use the tool to link up Azure Services and Azure DevOps

  • First we log the CLI tool into Azure. You can find the values required from Azure Portal, in the Subscription overview and App Registration overview. You create a password from ‘client and secrets’ section for the App Registration.

.aggregator-cli.exe logon.azure –subscription <sub-id> –client <client-id> –tenant <tenant-id> –password <pwd>

.aggregator-cli.exe logon.ado –url https://dev.azure.com/<org> –mode PAT –token <pat>

  • Now we can create the Instance of the Aggregator in  Azure

    Note: I had ling delays and timeout problems here due to what turned our to be a  poor WIFI link. The strange thing was it was not obviously failing WIFI but just unstable enough to cause issues. As soon as I swapped to Ethernet the problems went away.

    The basic form of the command is as follows, this will create a new resource group in Azure and then the required Web App, Storage, Application Insights etc. As this is  done using an ARM template so it is idempotent i.e. it can re run as many times as you wish, it will just update the Azure services if they already exist.

    .aggregator-cli.exe install.instance –verbose –name yourinstancename –location westeurope

  • When this completes, you can see the new resources in the Azure Portal, or check them with command line

    .aggregator-cli.exe list.instances

  • You next need to register your rules. You can register as many as you wish. A few samples are provided in the test folder in the downloaded ZIP, these are good for a quick tests, thought you will usually create your own for production use. When you add a rule, behind the scenes this creates an Azure Function with the same name as the rule.

    .aggregator-cli.exe add.rule –verbose –instance yourinstancename –name test1 –file testtest1.rule

  • Finally you map a rule to some event in Azure DevOps instance

    .aggregator-cli.exe map.rule –verbose –project yourproject –event workitem.updated –instance rfado –rule test1

And once all this done you should have a working system. If you are using the the test rules then quickest option to see it is working is to

  1. Go into the Azure Portal
  2. Find the created Resource Group
  3. Pick the App Service for the Azure Functions
  4. Pick the Function for the rule under test
  5. Pick the Monitor
  6. Pick Logs
  7. Open Live Metric
  8. You should see log entries when you perform the event on a work item you mapped to the function.

So I hope this helps my future self remember how get this tool setup quickly

How to do local template development for my Cross platform Release notes task

The testing cycle for Release Notes Templates can be slow, requiring a build and release cycle. To try to speed this process for users I have created a local test harness that allows the same calls to be made from a development machine as would be made within a build or release.

However, running this is not as simple was you might expect so please read the instruction before proceeding.

Setup and Build

  1. Clone the repo contain the Azure DevOps Extension.
  2. Change to the folder

    <repo root>ExtensionsXplatGenerateReleaseNotesV2testconsole

  3. Build the tool using NPM (this does assume Node is already installed)

    npm install
    npm run build

Running the Tool

The task the testconsole runs takes many parameters, and reads runtime Azure DevOps environment variable. These have to be passing into the local tester. Given the number, and the fact that most probably won’t need to be altered, they are provided in settings JSON file. Samples are provided for a build and a release. For details on these parameters see the task documentation

The only values not stored in the JSON files are the PATs required to access the REST API. This reduces the chance of them being copied onto source control by mistake.

Two PATs are potentially used.

  • Azure DevOps PAT (Required) – within a build or release this is automatically picked up. For this tool it must be provided
  • GitHub PAT – this is an optional parameter for the task, you only need to provide it if working with private GitHub repos as your code store. So usually this can be ignored.

Test Template Generation for a Build

To run the tool against a build

  1. In the settings file make sure the TeamFoundationCollectionUri, TeamProject and BuildID are set to the build you wish to run against, and that the ReleaseID is empty.
  2. Run the command

    node .GenerateReleaseNotesConsoleTester.js build-settings.json <your-Azure-DevOps-PAT> <Optional: your GitHub PAT>

  3. Assuming you are using the sample settings you should get an output.md file with your release notes.

Test Template Generation for a Release

To run the tool against a release is but more complex. This is because the logic looks back to see the most recent successful run. So if your release ran to completion you will get no notes as there has been no changes it it is the last successful release.

You have two options

  • Allow a release  to trigger, but cancel it. You can then use its ReleaseID to compare with the last release
  • Add a stage to your release this is skipped, only run on a manual request and use this as the comparison stage to look for difference

To run the tool

  1. In the settings file make sure the TeamFoundationCollectionUri, TeamProject, BuildID, EnvironmentName (as stage in your process), ReleaseID and releaseDefinitionId are set for the release you wish to run against.
  2. Run the command

    node .GenerateReleaseNotesConsoleTester.js release-settings.json <your-Azure-DevOps-PAT> <Optional: yourGitHub PAT>

  3. Assuming you are using the sample settings you should get an output.md file with your release notes.

Hope you find it useful