Authentication loops swapping organisations in Azure DevOps

I have recently been getting a problem swapping between different organisations in Azure DevOps. It happens when I swap between Black Mable ones and customer ones, where each is back by different Azure Active Directory (AAD) but I am using the same credentials; because I am either a member of that AAD or a guest.

The problem is I get into an authentication loop. It happens to be in Chrome, but you might find the same problem in other browsers.

It seems to be a recent issue, maybe related to MFA changes in AAD?

I used to be re-promoted for my ID when I swapped organisations in a browser tab, but not asked for further authentication

However, now the following happens

  • I login to an organisation without a problem e.g https://dev.azure.com/someorg using ID, password and MFA
  • In the same browser window, when I connect to another organisation e.g. https://dev.azure.com/someotherorg 
  • I am asked to pick an account, then there is the MFA challenge, but then go back to the login
  • …. and repeat.

The fix is to go in the browser tab to https://dev.azure.com. As you are already authenticated you will be able to sign out, then all is OK, you can login again.

The other options is to make even more use of Chrome People; one ‘person’ per customer, as opposed to my current usage on one ‘person’ per ID

You can’t use Azure DevOps Pipeline Gates to check services behind a firewall

I have recently be working on a release pipeline that deploys to a server behind a corporate firewall. This is done using an Azure DevOps private build agent and works fine.

As the service is a basic REST service and takes a bit of time to start-up I though a gate was a perfect way to pause the release pipeline until service was ready for the automated tests.

However, I hit a problem, the gates always failed as the internal server could not be resolved.

After a bit of thought I realised why. Gates as actual agentless tasks, they don’t run on the agent but on the server, so are outside the firewall. They could never connect to the private service without ports being opened, which was never going to happen.

So at this point in time I can’t use gates on this pipeline. Any similar logic to do the same job would have to be developed as scripts I can run on an agent.

Review of ‘Azure DevOps Server 2019 Cookbook’ – well worth getting

It always amazes me that people find time to write tech books whilst having a full time job. So given the effort I know it will have been, it is great to see an update to  Tarun Arora and Utkarsh Sigihalli’s book ‘Azure DevOps Server 2019 Cookbook’.

Azure DevOps Server 2019 Cookbook - Second Edition

I do like their format of ‘recipes’  that walk through common requirements. I find it particularly interesting that for virtually each recipes there is an associated Azure DevOps Extension that enhances the experience. It speaks well of the research the authors have done and the richness and variety of the 3rd party extensions in the Azure DevOps Marketplaces

I think because of this format there is something in this book for everyone, whether new to Azure DevOps Server 2019 or someone who has been around the product since the days of TFS 2005.

In my opinion, it is well worth having a copy on your shelf, whether physical or virtual

Azure DevOps Repos branch build policies not triggering when expected in PRs – Solved

I recently hit a problem with builds triggered by branch policies in Azure DevOps Repos. With the help of Microsoft I found out the problem and I thought it worth writing up uncase others hit the issue.

Setup

Folders

Assume you have a Git repo with source for the UI, backend Services and common code in sub folders

/ [root]
     UI
     Services
     Common

Branch Policies

On the Master branch there are a policies of running

  • one build for anything in the UI folder/project or common folder/project
  • and a different build for anything in the Services folder/project or common folder/project

These build were filtered by path using the filters

/UX; /Common
/Services; /Common

The Issue

I discovered the problem by doing the following

  • Create a PR for some work that effects the UI project
  • As expected the UI build triggers
  • Update the PR with a second commit for the Services code
  • The Service build is not triggered

The Solution

The fix was simple it turns out. Remove the spaces from the filter paths so they become

/UX;/Common
/Services;/Common

Once this was done the builds triggered as expected.

Thanks again to the Azure DevOps Product Group for the help

Regex issues in Node

I have been trying to use Regex to select a block of an XML based .NET Core CSPROJ file, and yes before you say know I could use XPATH, but why am not is another story.

I was trying to use the Regex

content.match(/<PropertyGroup>((.|\n)*)<\/PropertyGroup>/gmi)

The strange thing was this selection string worked in online Regex testers and in online Javascript IDEs, but failed inside my Node based Azure DevOps Pipeline extension.

After much experimentation I found that the following line worked

content.match(/<PropertyGroup>([\s\S]*?)<\/PropertyGroup>/gmi)


Well that a a good few hours of my life I won’t get back. No idea why Node handles the wildcards differently

A fix for Error: SignerSign() failed." (-2146958839/0x80080209) with SignTool.exe

I have spent too long recently trying to sign a UWP .MSIXBUNDLE generated from an Azure DevOps build using the SignTool.exe and our code signing certificate. I kept getting the error

Done Adding Additional Store
Error information: "Error: SignerSign() failed." (-2146958839/0x80080209)

From past experience, SignTool errors are usually due to the publisher details in the XML manifest files (in this case unpack the bundle with MakeAppx.exe and look in AppxMetadata\AppxBundleManifest.xml, and also check the manifest in the bundled .MSIX files) does not match the subject details for the PFX file being used for signing. 

Or so I thought…..

Turns out you can get this error too if you use the wrong version of the SignTool, but it give no clue to this fact.

So the top tip is …

Make sure you use the SignTool.exe from the same folder as the MakeAppx.exe tool. In  my case in “C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\”

Once I did this, after of course updating all the manifest files with the correct publisher details, I was able to sign my bundle as I wanted.

Migrating a GUI based build to YAML in Azure DevOps Pipelines

Introduction

I use Azure DevOps Pipelines for the build and release of my Azure DevOps Pipeline extensions, I previously detailed my process here .

For a good few months now YAML builds have been available. These provide the key advantage that the build is defined in a YAML text file that is stored with your product’s source code, thus allowing you to more easily track build changes. Also bulk editing becomes easier as a simple text editor can be used.

I have been putting off moving my current GUI based builds for as there is a bit of work, this post document then step.

Process

Getting the old build content

First I created a new branch in my local copy of my GitHub repo that stores the source for my extensions

I then created an empty file azure-pipelines-build.yaml the same folder as the root of the extension I was replacing the build for. I created the empty text file. I did this as the current create new build UI allows you to pick a file or a create one, but if you create one it gives you no control as to where or how it is named

In you existing build I then clicked the pipeline level ‘View YAML’

image 

Note:  Initially I found this link disabled, but if you click around the UI, into the task details, variables etc, it eventually becomes enabled. I have no idea why.

Copy this YAML into you newly created azure-pipelines-build.yaml file, committed the file and pushed it GitHub as the new branch.

Creating the YAML build

I then created a new YAML based build, picking in my case GitHub as the source host, the correct branch, and correct file.

This YAML contains the  core of what is needed, but the build was missing some some items such as triggers, build number and variable.

I added

  • the name (build number)
  • the PR triggers to the YAML

to the .YAML file, but decided to declare my variables as they contained secrets within the build definition in Azure DevOps.

The final YAML file was can be viewed here

What I fixed in passing

In the past I used to package up my extensions twice, once packaged as private (for testing) and once as public. This was due to the limitation of the Azure DevOps Marketplace and the release tasks I was using at the time. Whilst passing a took the chance to change to only building the public VSIX package, but updated my release pipeline process to dynamically inject the settings for private testing. This was done using the newer Azure DevOps Extensions Tasks.

As I side note I had to upgrade to these newer release tasks anyway as the older ones had ceased to work due to using old API calls

Swapping in the new build into the release process

To replace the old GUI build with the new YAML build I did the following

  • Renamed my old GUI build and disabled this (the disable is vital else it continues to be triggered by the GitHub PRs, even if the triggers are removed in the build)
  • Renamed my new YAML build to the old GUI build name (not vital, but it felt neater)
  • Updated my release pipeline to pick the new YAML build as opposed to the old GUI build. Even though the names were the same, their internal IDs are not, so this needs to be swapped. I made sure my ‘source alias’ did not change, so I did not have to make other changes to my release pipeline. 

Once this was done I triggered a new GitHub PR and everything worked as expects.

What Next

I have kept the old build about just in case there is a problem I have not spotted, but I intend to delete this soon.

I now need to make the same changes for all my other build. The only difference for from this process will be for builds that make use of Task Groups, such as all those for Node based extensions. Task Groups cannot be exported as YAML at this time, so I will have to manually rebuilding these steps in a text editor. So more prone to human error, but I think it needs to be done.

So a nice back burner project. I will probably update them as release new versions of extensions.

A task for documenting your Azure DevOps Pipeline extensions for YAML usage

I have posted in the past a quick script to generate some markdown documentation for the YAML usage of Azure DevOps Pipeline extensions. Well I decided that having this script as a task itself would be a good idea, so a wrote it, and please to say have just release it to the marketplace

The YAML Documenter task scans an extension’s vss-extension.json and task.json files to find the details it needs to build the markdown documentation on the YAML usage. It can also, optionally, copy the extension’s readme.md as the extensions primary documentation.

I am starting to use this extension, with my WIKIUpdater extension, in my release pipelines to make sure my extension’s GitHub WIki is up to date.

image

It is going to take a bit of work to update all my pipelines, but the eventual plan is to use the YAML document generator in the builds, adding the readme and YAML markdown files to the build as artefacts. Then deploying these files to the wiki in a later stage of the pipeline.

Hope some of you find it of use.

Programmatically adding User Capabilities to Azure DevOps Agents

I am automating the process by which we keep our build agent up to date. The basic process is to use a fork of the standard Microsoft Azure DevOps Pipeline agent that has the additional code included we need, notably Biztalk.

Once I have the Packer created VM up and running, I need to install the agent. This is well document, just run .\config.cmd –help for details. However, there is no option to add user capabilities to the agent.

I know I could set them via environment variables, but I don’t want the same user capabilities on each agent on a VM (we use multiple agents on a single VM).

There was no documented Azure DevOps API I could find to add capabilities, but a bit of hacking around with Chrome Dev tools and Postman got me a solution, which I have provided a GIST

Azure Pipeline YAML support on VSCode

A major problem when moving from the graphic editing of Azure Pipeline builds to YAML has been the difficulty in knowing the options available, and of course making typos.

Microsoft have just released a VSCode extension to help address this problem – it is called Azure Pipelines

I have yet to give it a really good workout, but first impressions are good.

It does not remove the need for good documentation of task options, there is a need for my script to generate YAML documentation from a task.json file, but anything extra to ease editing helps.