Fix for ‘System.BadImageFormatException’ when running x64 based tests inside a Azure DevOps Release

This is one of those blog posts I write to remind my future self how I fixed a problem.

The Problem

I have a release that installs VSTest and runs some integration tests that target .NET 4.6 x64. All these tests worked fine in Visual Studio. However, I got the following errors for all tests when they were run in a release

2020-04-23T09:30:38.7544708Z vstest.console.exe "C:\agent\_work\r1\a\PaymentServices\drop\testartifacts\PaymentService.IntegrationTests.dll"

2020-04-23T09:30:38.7545688Z /Settings:"C:\agent\_work\_temp\uxykzf03ik2.tmp.runsettings"

2020-04-23T09:30:38.7545808Z /Logger:"trx"

2020-04-23T09:30:38.7545937Z /TestAdapterPath:"C:\agent\_work\r1\a\PaymentServices\drop\testartifacts"

2020-04-23T09:30:39.2634578Z Starting test execution, please wait...

2020-04-23T09:30:39.4783658Z A total of 1 test files matched the specified pattern.

2020-04-23T09:30:40.8660112Z   X Can_Get_MIDs [521ms]

2020-04-23T09:30:40.8684249Z   Error Message:

2020-04-23T09:30:40.8684441Z    Test method PaymentServices.IntegrationTests.ControllerMIDTests.Can_Get_MIDs threw exception:

2020-04-23T09:30:40.8684574Z System.BadImageFormatException: Could not load file or assembly 'PaymentServices, Version=, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An attempt was made to load a program with an incorrect format.

2020-04-23T09:30:40.8684766Z   Stack Trace:

2020-04-23T09:30:40.8684881Z       at PaymentServices.IntegrationTests.ControllerMIDTests.Can_Get_MIDs()

2020-04-23T09:30:40.9038788Z Results File: C:\agent\_work\_temp\TestResults\svc-devops_SVRHQAPP027_2020-04-23_10_30_40.trx

2020-04-23T09:30:40.9080344Z Total tests: 22

2020-04-23T09:30:40.9082348Z      Failed: 22

2020-04-23T09:30:40.9134858Z ##[error]Test Run Failed.


I needed to tell vstest.console.exe to run x64 as opposed to it’s default of x32. This can be done with a command line override –platform:x64


And more enriching the data available in my Azure DevOps Pipelines Cross Platform Release Notes Task

I have today released another enrichment to the dataset available in my Cross Platform Release Notes Azure Pipeline Task. It now returns an extra array of data that links work items and commits to build artifacts.

So your reporting objects are:

Array Objects

  • workItems – the array of all work item associated with the release
  • commits – the array of all commits associated with the release
  • pullRequests – the array of all PRs referenced by the commits in the release
  • The new one – builds – the array of the build artifacts that CS and WI are associated with. Note that this is a object with three properties
    • build – the build details
    • commits – the commits associated with this build
    • workitems – the work items associated with the build

Release objects (only available in a release)

  • releaseDetails – the release details of the release that the task was triggered for.
  • compareReleaseDetails – the the previous successful release that comparisons are being made against

Build objects

  • buildDetails – if running in a build, the build details of the build that the task is running in. If running in a release it is the build that triggered the release.

Note: To dump all possible values use the form {{json propertyToDump}} this runs a custom Handlebars extension to do the expansion

It is important to realised that these arrays are only available using the Handlebars form of templating. You can find sample here

Further enriching the data available in my Azure DevOps Pipelines Cross Platform Release Notes Task

I recently post about Enriching the data available in my Azure DevOps Pipelines Cross Platform Release Notes Task by adding Pull Request information. Well, that first release was fairly limited only working for PR validation builds, so I have made more improvements and shipped a newer version.

The task now will, as well as checking for PR build trigger, try to associate the commits associated with a build/release pipeline to any completed PRs in the repo. This is done using the Last Merge Commit ID, and from my tests seems to work for the various types of PR e.g. squash, merge, rebased and semi-linear.

The resultant set of PRs are made available to the release notes template processor as an array. However, there is a difference between the existing arrays for Work Items and Commits and the new one for Pull requests. The new one is only available if you are using the new Handlebars based templating mode.

You would add a block in the general form….

{{#forEach pullRequests}}
{{#if isFirst}}### Associated Pull Requests (only shown if  PR) {{/if}}
*  **PR {{}}**  {{this.title}}

The reason I have chosen to only support Handlebars is it make the development so much easier and provides a more flexible solution, given all the Handlebar helpers available. I think this might be the first tentative step towards deprecating my legacy templating solution in favour of only shipping Handlebars support.

Swapping my Azure DevOps Pipeline Extensions release process to use Multistage YAML pipelines

In the past I have documented the build and release process I use for my Azure DevOps Pipeline Extensions and also detailed how I have started to move the build phases to YAML.

Well now I consider that multistage YAML pipelines are mature enough to allow me to do my whole release pipeline in YAML, hence this post.


My pipeline performs a number of stages, you can find a sample pipeline here. Note that I have made every effort to extract variables into variable groups to aid reuse of the pipeline definition. I have added documentation as to where variable are stored and what they are used for.

The stages are as follows


The build phase does the following

  • Updates all the TASK.JSON files so that the help text has the correct version number
  • Calls a YAML template (build-Node-task) that performs all the tasks to transpile a TypeScript based task – if my extension contained multiple tasks this template would be called a number of time
    • Get NPM packages
    • Run Snyk to check for vulnerabilities – if any vulnerabilities are found the build fails
    • Lint and Transpile the TypeScript – if any issue are found the build fails
    • Run any Unit test and publish results – if any test fail the build fails
    • Package up the task (remove dev dependencies)
  • Download the TFX client
  • Package up the Extension VSIX package and publish as a pipeline artifact.


The private phase does the following

  • Using another YAML template (publish-extension) publish the extension to the Azure DevOps Marketplace, but with flags so it is private and only assessible to my account for testing
    • Download the TFX client
    • Publishes the Extension to the Marketplace

This phase is done as a deployment job and is linked to an environment,. However, there are no special approval requirements are set on this environment. This is because I am happy for the release to be done to the private instance assuming the build phase complete without error.


This is where the pipeline gets interesting. The test phase does the following

  • Runs any integration tests. These could be anything dependant on the extension being deployed. Unfortunately there is no option at present in multistage pipeline for a manual task to say ‘do the manual tests’, but you could simulate similar by sending an email or the like.

The clever bit here is that I don’t want this stage to run until the new private version of the extension has been published and is available; there can be a delay between TFX saying the extension is published and it being downloadable by an agent. This can cause a problem in that you think you are running tests against a different version of the extension to one you have. To get around this problem I have implemented a check on the environment this stage’s deployment job is linked to. This check runs an Azure Function to check the version of the extension in the Marketplace. This is exactly the same Azure Function I already used in my UI based pipelines to perform the same job.

The only issue here is that this Azure Function is used as an exit gate in my UI based pipelines; to not allow the pipeline to exit the private stage until the extension is publish. I cannot do this in a multistage YAML pipeline as environment checks are only done on entry to the environment. This means I have had to use an extra Test stage to associate the entry check with. This was setup as follows

  • Create a new environment
  • Click the ellipse (…) and pick ‘approvals and checks’
  • Add a new Azure Function check
  • Provide the details, documented in my previous post, to link to your Azure Function. Note that you can, in the ’control options’ section of the configuration, link to a variable group. This is a good place to store all the values, you need to provide
    • URL of the Azure Function
    • Key to us the function
    • The function header
    • The body – this one is interesting. You need to provide the build number and the GUID of a task in the extension for my Azure Function. It would be really good if both of these could be picked up from the pipeline trying to use the environment. This would allow a single ‘test’ environment to be created for use by all my extensions, in the same way there are only a single ‘private’ and ‘public’ environment. However, there is a problem, the build number is picked up OK, but as far as I can see I cannot access custom pipeline variables, so cannot get the task GUID I need dynamically. I assume this is because this environment entry check is run outside of the pipeline. The only solution  can find is to place the task GUID as a hard coded value in the check declaration (or I suppose in the variable group). The downside of this is it means I have to have an environment dedicated to each extension, each with a different task GUID. Not perfect, but not too much of a problem
    • In the Advanced check the check logic
    • In control options link to the variable group contain any variables used.


The documentation stage again uses a template (generate-wiki-docs) and does the following


The public stage is also a deployment job and linked to an environment. This environment has an approval set so I have to approve any release of the public version of the extension.

As well as doing the same as private stage this stage does the following


It took a bit of trial and error to get this going, but I think I have a good solution now. The fact that the bulk of the work is done using shared templates means I should get good reuse of the work I have done. I am sure I will be able to improve the template as time goes on but it is a good start

My Azure DevOps Pipeline is not triggering on a GitHub Pull request – fixed

I have recently hit a problem that some of my Azure DevOps YAML pipelines, that I use to build my Azure DevOps Pipeline Extensions, are not triggering on a new PR being created on GitHub.

I did not get to the bottom of why this is happening, but I found a fix.

  • Check and of make a note of any UI declared variables in your Azure DevOps YAML Pipeline that is not triggering
  • Delete the pipeline
  • Re-add the pipeline, linking to the YAML file hosted on GitHub. You might be asked to re-authorise the link between Azure DevOps Pipelines and GitHub.
  • Re-enter any variables that are declared via the Pipelines UI and save the changes

Your pipeline should start to be triggered again

Enriching the data available in my Azure DevOps Pipelines Cross Platform Release Notes Task

A common request for my Generate Release Notes Tasks is to enrich the data available beyond basic build, work item and commit/changeset details. I have resisted these requests as it felt like a never ending journey to start. However, I have now relented and added the option to see any pull request information available.

This feature is limited, you obviously have to be using artifacts that linked to a Git repo, and also the Git repo have to on an Azure DevOps hosted repository. This won’t meet everyone’s needs but it is a start.

What was already available

Turns out there was already a means to get a limited set of PR details from a build. You used the form

**Build Trigger PR Number**: ${buildDetails.triggerInfo['pr.number']}

or in handlebars format

**Build Trigger PR Number**: {{lookup buildDetails.triggerInfo 'pr.number'}} 

The improvements

That said I have improved the options. There is now a new `prDetails` object available to the template.

If you use the dump option


You can see the fields available

     "repository": {
         "id": "bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59",
         "name": "VSTSBuildTaskValidation",
         "url": "",
         "project": {
             "id": "670b3a60-2021-47ab-a88b-d76ebd888a2f",
             "name": "GitHub",
             "description": "A container for GitHub CI/CD processes",
             "url": "",
             "state": "wellFormed",
             "revision": 411511726,
             "visibility": 2,
             "lastUpdateTime": "2019-10-10T20:35:51.85Z"
         "size": 9373557,
         "remoteUrl": "",
         "sshUrl": "",
         "webUrl": ""
     "pullRequestId": 4,
     "codeReviewId": 4,
     "status": 1,
     "createdBy": {
         "displayName": "Richard Fennell (Work MSA)",
         "url": "",
         "_links": {
             "avatar": {
                 "href": ""
         "id": "b1fce0e9-fbf4-4202-bc09-a290def3e98b",
         "uniqueName": "",
         "imageUrl": "",
         "descriptor": "aad.NzQzY2UyODUtN2Q0Ny03YjNkLTk0ZGUtN2Q0YjA1ZGE5NDdj"
     "creationDate": "2020-04-04T10:44:59.566Z",
     "title": "Added test.txt",
     "description": "Added test.txt",
     "sourceRefName": "refs/heads/branch2",
     "targetRefName": "refs/heads/master",
     "mergeStatus": 3,
     "isDraft": false,
     "mergeId": "f76a6556-8b4f-44eb-945a-9350124f067b",
     "lastMergeSourceCommit": {
         "commitId": "f43fa4de163c3ee0b4f17b72a659eac0d307deb8",
         "url": ""
     "lastMergeTargetCommit": {
         "commitId": "829ab2326201c7a5d439771eef5a57f58f94897d",
         "url": ""
     "lastMergeCommit": {
         "commitId": "53f393cae4ee3b901bb69858c4ee86cc8b466d6f",
         "author": {
             "name": "Richard Fennell (Work MSA)",
             "email": "",
             "date": "2020-04-04T10:44:59.000Z"
         "committer": {
             "name": "Richard Fennell (Work MSA)",
             "email": "",
             "date": "2020-04-04T10:44:59.000Z"
         "comment": "Merge pull request 4 from branch2 into master",
         "url": ""
     "reviewers": [],
     "url": "",
     "supportsIterations": true,
     "artifactId": "vstfs:///Git/PullRequestId/670b3a60-2021-47ab-a88b-d76ebd888a2f%2fbebd0ae2-405d-4c0a-b9c5-36ea94c1bf59%2f4"

In templates this new object could be is used

**PR Title **: ${prDetails.title}

or in handlebars format.

**PR Details**: {{prDetails.title}}

It will be interesting to here feedback from the real world as opposed to test harnesses

Experiences setting up Azure Active Directory single sign-on (SSO) integration with GitHub Enterprise


GitHub is a great system for individuals and OSS communities for both public and private project. However, corporate customers commonly want more control over their system than the standard GitHub offering. It is for this reason GitHub offers  GitHub Enterprise.

For most corporates, the essential feature that GitHub Enterprise offers is the use Single Sign On (SSO) i.e. allowing users to login to GitHub using their corporate directory accounts.

I wanted to see how easy this was to setup when you are using Azure Active Directory (AAD).

Luckily there is a step by step tutorial from Microsoft on how to set this up. Though, I would say that though detailed this tutorial has a strange structure in that it shows the default values not the correct values. Hence, the tutorial requires close reading, don’t just look at the pictures!

Even with close reading, I still hit a problem, all of my own making, as I went through this tutorial.

The Issue – a stray / in a URL

I entered all the AAD URLs and certs as instructed (or so I thought) by the tutorial into the Security page of GitHub Enterprise.

When I pressed the ‘Validate’ button in GitHub, to test the SSO settings, I got an error

‘The client has not listed any permissions for ‘AAD Graph’ in the requested permissions in the client’s application registration’

This sent me shown a rabbit hole looking at user permissions. That wasted a lot of time.

However, it turns out the issue was that I had a // in a URL when it should have been a  /. This was because I had made a cut and paste error when editing the tutorial’s sample URL and adding my organisation details.

Once I fixed this typo the validation worked, I was able to complete the setup and then I could to invite my AAD users to my GitHub Enterprise organisation.


So the summary is, if you follow the tutorial setting up SSO from AAD to GitHub Enterprise is easy enough to do, just be careful of over the detail.

A major new feature for my Cross-platform Release Notes Azure DevOps Pipelines Extension–Handlebars Templating Support

I recently got a very interesting PR for my Cross-platform Release Notes Azure DevOps Pipelines Extension from Kenneth Scott. He had added a new templating engine to the task, Handlebars.

Previous to this PR the templating in the task was done with a line by line evaluation of a template that used my own mark-up. This method worked but has limitations, mostly due to the line by line evaluation model.  With the Kenneth’s PR the option was added to write your templates in Handlebars, or stay with my previous templating engine.

Using Handlebars

If you use Handlebars, the template becomes something like

## Notes for release  {{}}    
**Release Number**  : {{}}
**Release completed** : {{releaseDetails.modifiedOn}}     
**Build Number**: {{}}
**Compared Release Number**  : {{}}    

### Associated Work Items ({{workItems.length}})
{{#each workItems}}
*  **{{}}**  {{lookup this.fields 'System.Title'}}
   - **WIT** {{lookup this.fields 'System.WorkItemType'}} 
   - **Tags** {{lookup this.fields 'System.Tags'}}

### Associated commits ({{commits.length}})
{{#each commits}}
* ** ID{{}}** 
   -  **Message:** {{this.message}}
   -  **Commited by:** {{}} 

The whole template is evaluated by the Handlebars engine using its own mark-up to provide a means for looping across arrays and the like.

This seemed a great enhancement to the task. However, we soon realised that it could be better. Handlebars is extensible, so why not allow the extensibility to be used?

Using Handlebars Extensions

I have added extensibility in two ways. Firstly I have also added support for the common Handlebar-Helpers extensions, this added over 150 helpers. These are just accessed in a template as follows

## To confirm the handbars-helpers is work
The year is {{year}} 
We can capitalize "foo bar baz" {{capitalizeAll "foo bar baz"}}

I have also added the ability to provide a block of JavaScript as a task parameter is that is loaded as a custom Handlebars extension. So if you add the following block in the tasks customHandlebarsExtensionCode parameter.

module.exports = {foo: function () {return 'Returns foo';}};

You can access in the templates as

## To confirm our custom extension works
We can call our custom extension {{foo}}

It will be interesting to see how popular this alternative way of templating will be.

Where did all my test results go?


I recently tripped myself up whist adding SonarQube analysis to a rather complex Azure DevOps build.

The build has two VsTest steps, both were using the same folder for their test result files. When the first VsTest task ran it created the expected .TRX and .COVERAGE files and then published its results to Azure DevOps, but when the second VsTest task ran it over wrote this folder, deleting the files already present, before it generated and published it results.

This meant that the build itself had all the test results published, but when SonarQube looked for the files for analysis only the second set of test were present, so its analysis was incorrect.


The solution was easy, use different folders for each set of test results.

This gave me a build, the key items are shown below, where one VsTest step does not overwrite the previous results before they can be processed by any 3rd party tasks such as SonarQube.

- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4
   displayName: 'Prepare analysis on SonarQube'
     SonarQube: Sonarqube
     projectKey: 'Services'
     projectName: 'Services'
     projectVersion: '$(major).$(minor)'
     extraProperties: |
      # Additional properties that will be passed to the scanner,

… other build steps

- task: VSTest@2
   displayName: 'VsTest – Internal Services'
     testAssemblyVer2: |
     searchFolder: '$(System.DefaultWorkingDirectory)/src/Services'
     resultsFolder: '$(System.DefaultWorkingDirectory)\TestResultsServices'
     overrideTestrunParameters: '-DeploymentEnabled false'
     codeCoverageEnabled: true
     testRunTitle: 'Services Unit Tests'
     diagnosticsEnabled: True
   continueOnError: true

- task: VSTest@2
   displayName: 'VsTest - External'
     testAssemblyVer2: |
     searchFolder: '$(System.DefaultWorkingDirectory)/src/ExternalServices'
     resultsFolder: '$(System.DefaultWorkingDirectory)\TestResultsExternalServices'
     vsTestVersion: 15.0
     codeCoverageEnabled: true
     testRunTitle: 'External Services Unit Tests'
     diagnosticsEnabled: True
   continueOnError: true

- task: BlackMarble.CodeCoverage-Format-Convertor-Private.CodeCoverageFormatConvertor.CodeCoverage-Format-Convertor@1
   displayName: 'CodeCoverage Format Convertor'
     ProjectDirectory: '$(System.DefaultWorkingDirectory)'

- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4
   displayName: 'Run Code Analysis'

- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4
   displayName: 'Publish Quality Gate Result'