Azure DevOps Repos branch build policies not triggering when expected in PRs – Solved

I recently hit a problem with builds triggered by branch policies in Azure DevOps Repos. With the help of Microsoft I found out the problem and I thought it worth writing up uncase others hit the issue.

Setup

Folders

Assume you have a Git repo with source for the UI, backend Services and common code in sub folders

/ [root]
     UI
     Services
     Common

Branch Policies

On the Master branch there are a policies of running

  • one build for anything in the UI folder/project or common folder/project
  • and a different build for anything in the Services folder/project or common folder/project

These build were filtered by path using the filters

/UX; /Common
/Services; /Common

The Issue

I discovered the problem by doing the following

  • Create a PR for some work that effects the UI project
  • As expected the UI build triggers
  • Update the PR with a second commit for the Services code
  • The Service build is not triggered

The Solution

The fix was simple it turns out. Remove the spaces from the filter paths so they become

/UX;/Common
/Services;/Common

Once this was done the builds triggered as expected.

Thanks again to the Azure DevOps Product Group for the help

Migrating a GUI based build to YAML in Azure DevOps Pipelines

Introduction

I use Azure DevOps Pipelines for the build and release of my Azure DevOps Pipeline extensions, I previously detailed my process here .

For a good few months now YAML builds have been available. These provide the key advantage that the build is defined in a YAML text file that is stored with your product’s source code, thus allowing you to more easily track build changes. Also bulk editing becomes easier as a simple text editor can be used.

I have been putting off moving my current GUI based builds for as there is a bit of work, this post document then step.

Process

Getting the old build content

First I created a new branch in my local copy of my GitHub repo that stores the source for my extensions

I then created an empty file azure-pipelines-build.yaml the same folder as the root of the extension I was replacing the build for. I created the empty text file. I did this as the current create new build UI allows you to pick a file or a create one, but if you create one it gives you no control as to where or how it is named

In you existing build I then clicked the pipeline level ‘View YAML’

image 

Note:  Initially I found this link disabled, but if you click around the UI, into the task details, variables etc, it eventually becomes enabled. I have no idea why.

Copy this YAML into you newly created azure-pipelines-build.yaml file, committed the file and pushed it GitHub as the new branch.

Creating the YAML build

I then created a new YAML based build, picking in my case GitHub as the source host, the correct branch, and correct file.

This YAML contains the  core of what is needed, but the build was missing some some items such as triggers, build number and variable.

I added

  • the name (build number)
  • the PR triggers to the YAML

to the .YAML file, but decided to declare my variables as they contained secrets within the build definition in Azure DevOps.

The final YAML file was can be viewed here

What I fixed in passing

In the past I used to package up my extensions twice, once packaged as private (for testing) and once as public. This was due to the limitation of the Azure DevOps Marketplace and the release tasks I was using at the time. Whilst passing a took the chance to change to only building the public VSIX package, but updated my release pipeline process to dynamically inject the settings for private testing. This was done using the newer Azure DevOps Extensions Tasks.

As I side note I had to upgrade to these newer release tasks anyway as the older ones had ceased to work due to using old API calls

Swapping in the new build into the release process

To replace the old GUI build with the new YAML build I did the following

  • Renamed my old GUI build and disabled this (the disable is vital else it continues to be triggered by the GitHub PRs, even if the triggers are removed in the build)
  • Renamed my new YAML build to the old GUI build name (not vital, but it felt neater)
  • Updated my release pipeline to pick the new YAML build as opposed to the old GUI build. Even though the names were the same, their internal IDs are not, so this needs to be swapped. I made sure my ‘source alias’ did not change, so I did not have to make other changes to my release pipeline. 

Once this was done I triggered a new GitHub PR and everything worked as expects.

What Next

I have kept the old build about just in case there is a problem I have not spotted, but I intend to delete this soon.

I now need to make the same changes for all my other build. The only difference for from this process will be for builds that make use of Task Groups, such as all those for Node based extensions. Task Groups cannot be exported as YAML at this time, so I will have to manually rebuilding these steps in a text editor. So more prone to human error, but I think it needs to be done.

So a nice back burner project. I will probably update them as release new versions of extensions.

Programmatically adding User Capabilities to Azure DevOps Agents

I am automating the process by which we keep our build agent up to date. The basic process is to use a fork of the standard Microsoft Azure DevOps Pipeline agent that has the additional code included we need, notably Biztalk.

Once I have the Packer created VM up and running, I need to install the agent. This is well document, just run .\config.cmd –help for details. However, there is no option to add user capabilities to the agent.

I know I could set them via environment variables, but I don’t want the same user capabilities on each agent on a VM (we use multiple agents on a single VM).

There was no documented Azure DevOps API I could find to add capabilities, but a bit of hacking around with Chrome Dev tools and Postman got me a solution, which I have provided a GIST

Azure Pipeline YAML support on VSCode

A major problem when moving from the graphic editing of Azure Pipeline builds to YAML has been the difficulty in knowing the options available, and of course making typos.

Microsoft have just released a VSCode extension to help address this problem – it is called Azure Pipelines

I have yet to give it a really good workout, but first impressions are good.

It does not remove the need for good documentation of task options, there is a need for my script to generate YAML documentation from a task.json file, but anything extra to ease editing helps.

Keeping Azure DevOps organisations inherited process templates in sync

The problem

If you are like me for historic reasons you have multiple Azure DevOps organisations (instances) backed by the same Azure Active Directory (AAD). In my case for example: one was created when Azure DevOps was first released as TFSPreview.com and another is from our migration from on-prem TFS using the DB Migration Tools method; and I have others.

I make active use of all of these for different purposes, though one is primary with the majority of work done on it, and so I want to make sure the inherited process templates are the same on each of them. Using the primary organisation as the master customisation.

Note I have already converted all my old on-premises XML process models to inherited process templates.

There is no out the box way to do keep processes in syncs, but it is possible using a few tools. The main one is the Microsoft Process Migrator for Node on GitHub.

The Solution

Firstly I cloned the Microsoft Process Migrator and built it as per the instructions on the repo.

I created a config file and then ran the tool. On one organisation it ran fine. However on another I had errors like:

[ERROR] [2018-11-26T14:35:44.880Z] Process import validation failed. Process with same name already exists on target account.
[ERROR] [2018-11-26T14:39:54.206Z] Import failed, see log file for details. Create field ‘Location’ failed, see log for details

This was because I had in the past manually duplicated the inherited process template onto this organisation, so there was a process with the same name and fields of the same names.

The first error was easy to fix, import the template with a new (temporary) name.

The second is more problematic. I had two choice

As I only had a few duplicated unused fields on a single organisation I picked the former. If I had many organisations to sort out I would picked the latter.

So my process ended up being

  1. Run the Microsoft Process Migrator to migrate ‘My Process’ on the source organisation to ‘My Process 1’ on the target organisation
  2. It gave an error, providing the name of the duplicated field
  3. I checked on the target organisation using a work item query that the field was empty or only had defaulted data (if it had not been I would have used Martin’s tool to migrate the data to a temporary field and then deleted the problem field, moving the data back to the correct field from the temporary field when the import of the process template was completed)
  4. I deleted the field from the work item type that referenced it
  5. I deleted the field
  6. I deleted the process template ‘My Process 1’, a failed import leaves a half created process
  7. I went back to step 1 and repeated until the import completed without error
  8. I tested my migrated inherited process was OK
  9. On the target organisation I then renamed ‘My Process’ to ‘My Process – Old’
  10. I then renamed ‘My Process 1’ to ‘My Process’
  11. In my case I also made ‘My Process’ as the default, you might not do this if another process is the default, but step 13 does require the process template is not the default
  12. I moved all the team projects using the process template now called ‘My Process – Old’ to ‘My Process’
  13. I was then able to delete the process template ‘My Process – Old’ as it has no associated team projects and was not the default

As I customise my primary organisation’s process templates I can repeat this process to keep the processes in sync between organisations.  Note that in future migrations I won’t have to do steps 2..6 as there are no manually created duplicated fields. So it should be more straight forward.

So a valid solution until any similar functionality is built into Azure DevOps, and there is no sign of that on the roadmap.

Azure DevOps Services & Server Alerts DSL – an alternative to TFS Aggregator?

Whilst listening to a recent  Radio TFS it was mentioned that TFS Aggregator uses the C# SOAP based Azure DevOps APIs; hence needed a major re-write as these APIs are being deprecated.

Did you know that there was a REST API alternative to TFS Aggregator?

My Azure DevOps Services & Server Alerts DSL is out there, and has been for a while, but I don’t think used by many people. It aims to do the same as TFS Aggregator, but is based around Python scripting.

However, I do have to say it is more limited in flexibility as it has only been developed for my (and a few of my clients needs), but its an alternative that is based on the REST APIs. 

Scripts are of the following form, this one changes the state of a work item if all it children are done

import sys
# Expect 2 args the event type and a value unique ID for the wi
if sys.argv[0] == "workitem.updated" : 
    wi = GetWorkItem(int(sys.argv[1]))
    parentwi = GetParentWorkItem(wi)
    if parentwi == None:
        LogInfoMessage("Work item '" + str(wi.id) + "' has no parent")
    else:
        LogInfoMessage("Work item '" + str(wi.id) + "' has parent '" + str(parentwi.id) + "'")

        results = [c for c in GetChildWorkItems(parentwi) if c["fields"]["System.State"] != "Done"]
        if  len(results) == 0 :
            LogInfoMessage("All child work items are 'Done'")
            parentwi["fields"]["System.State"] = "Done"
            UpdateWorkItem(parentwi)
            msg = "Work item '" + str(parentwi.id) + "' has been set as 'Done' as all its child work items are done"
            SendEmail("richard@blackmarble.co.uk","Work item '" + str(parentwi.id) + "' has been updated", msg)
            LogInfoMessage(msg)
        else:
            LogInfoMessage("Not all child work items are 'Done'")
else:
	LogErrorMessage("Was not expecting to get here")
	LogErrorMessage(sys.argv)

I have recently done a fairly major update to the project. The key changes are:

  • Rename of project, repo, and namespaces to reflect Azure DevOps (the namespace change is a breaking change for existing users)
  • The scripts that are run can now be
    • A fixed file name for the web instance running the service
    • Based on the event type sent to the service
    • Use the subscription ID, thus allowing many scripts (new)
  • A single instance of the web site running the events processor can now handle calls from many Azure DevOps instances.
  • Improved installation process on Azure (well at least tried to make the documentation clearer and sort out a couple of MSDeploy issues)

Full details are on the project can be seen on the solutions WIKI, maybe you will find it of use. Let me know if the documentation is good enough

YAML documentation for my Azure Pipeline Tasks (and how I generated it)

There is a general move in Azure DevOps Pipelines to using YAML, as opposed to the designer, to define your pipelines. This is particularly enforced when using them via the new GitHub Marketplace Azure Pipelines method where YAML appears to be the only option.

This has shown up a hole in my Pipeline Tasks documentation, I had nothing on YAML!

So I have added a YAML usage page for each set of tasks in each of my extensions e.g the file utilities tasks.

Now, as are most developers, I am lazy. I was not going to type all that information. So I wrote a script to generate the markdown from respective task.json files in the repo. Now this script will need some work for others to use as it relies on some special handling due to quirks of my directory structure, but I hope it will be of use to others.

Microsoft post root cause analysis on recent Azure DevOps Issues

Azure DevOps has had some serious issue over the past couple of weeks with availability here in Europe.

A really good open and detailed root cause analysis has just been posted by the Azure DevOps team at Microsoft. It also covers the mitigations they are putting place to make sure this same issues do not occur again.

We all have to remember that the cloud is not magic. Cloud service providers will have problems like any on-premise services; but trying to hide them does nothing to build confidence. So I for one applaud posts like this. I just wish all cloud service providers were as open when problem occur.

Managing PATs on Azure DevOps just got loads clearer

It may have passed you by, it had me as I had not created a PAT for a while, but managing custom security for PATs in Azure DevOps is much easier since Sprint 140.

You now get some help to pick the correct ‘limited’ rights set by the simple grouping of rights.

image

We just need some more detailed documentation on what each option actually maps to permissions wise now to complete the picture

Using Paths in PR Triggers on an Azure DevOps Pipelines Builds

When I started creating OSS extensions for Azure DevOps Pipelines (starting on TFSPreview, then VSO, then VSTS and now named Azure DevOps) I made the mistake of putting all my extensions in a single GitHub repo. I thought this would make life easier, I was wrong, it should have been a repo per extension.

I have considered splitting the GitHub repo, but as a number of people have forked it, over 100 at the last count, I did not want to start a chain of chaos for loads of people.

This initial choice has meant that until very recently I could not use the Pull Request triggers in Azure DevOps Pipelines against my GitHub repo. This was because all builds associated with the repo triggered on any extension PR. So, I had to trigger builds manually, providing the branch name by hand. A bit of a pain, and prone to error.

I am pleased to say that with the roll out of Sprint 140 we now get the option to add a path filter to PR triggers on builds linked to GitHub repo; something we have had for Azure DevOps hosted Git repos since Sprint 126.

So now my release process is improved. If I add a path filter as shown below, my build and hence release process trigger on a PR just as I need.

image

It is just a shame that the GitHub PR only checks the build, not the whole release, before saying all is OK. Hope we see linking to complete Azure DevOps Pipelines in the future.