Programmatically adding User Capabilities to Azure DevOps Agents

I am automating the process by which we keep our build agent up to date. The basic process is to use a fork of the standard Microsoft Azure DevOps Pipeline agent that has the additional code included we need, notably Biztalk.

Once I have the Packer created VM up and running, I need to install the agent. This is well document, just run .\config.cmd –help for details. However, there is no option to add user capabilities to the agent.

I know I could set them via environment variables, but I don’t want the same user capabilities on each agent on a VM (we use multiple agents on a single VM).

There was no documented Azure DevOps API I could find to add capabilities, but a bit of hacking around with Chrome Dev tools and Postman got me a solution, which I have provided a GIST

Keeping Azure DevOps organisations inherited process templates in sync

The problem

If you are like me for historic reasons you have multiple Azure DevOps organisations (instances) backed by the same Azure Active Directory (AAD). In my case for example: one was created when Azure DevOps was first released as TFSPreview.com and another is from our migration from on-prem TFS using the DB Migration Tools method; and I have others.

I make active use of all of these for different purposes, though one is primary with the majority of work done on it, and so I want to make sure the inherited process templates are the same on each of them. Using the primary organisation as the master customisation.

Note I have already converted all my old on-premises XML process models to inherited process templates.

There is no out the box way to do keep processes in syncs, but it is possible using a few tools. The main one is the Microsoft Process Migrator for Node on GitHub.

The Solution

Firstly I cloned the Microsoft Process Migrator and built it as per the instructions on the repo.

I created a config file and then ran the tool. On one organisation it ran fine. However on another I had errors like:

[ERROR] [2018-11-26T14:35:44.880Z] Process import validation failed. Process with same name already exists on target account.
[ERROR] [2018-11-26T14:39:54.206Z] Import failed, see log file for details. Create field ‘Location’ failed, see log for details

This was because I had in the past manually duplicated the inherited process template onto this organisation, so there was a process with the same name and fields of the same names.

The first error was easy to fix, import the template with a new (temporary) name.

The second is more problematic. I had two choice

As I only had a few duplicated unused fields on a single organisation I picked the former. If I had many organisations to sort out I would picked the latter.

So my process ended up being

  1. Run the Microsoft Process Migrator to migrate ‘My Process’ on the source organisation to ‘My Process 1’ on the target organisation
  2. It gave an error, providing the name of the duplicated field
  3. I checked on the target organisation using a work item query that the field was empty or only had defaulted data (if it had not been I would have used Martin’s tool to migrate the data to a temporary field and then deleted the problem field, moving the data back to the correct field from the temporary field when the import of the process template was completed)
  4. I deleted the field from the work item type that referenced it
  5. I deleted the field
  6. I deleted the process template ‘My Process 1’, a failed import leaves a half created process
  7. I went back to step 1 and repeated until the import completed without error
  8. I tested my migrated inherited process was OK
  9. On the target organisation I then renamed ‘My Process’ to ‘My Process – Old’
  10. I then renamed ‘My Process 1’ to ‘My Process’
  11. In my case I also made ‘My Process’ as the default, you might not do this if another process is the default, but step 13 does require the process template is not the default
  12. I moved all the team projects using the process template now called ‘My Process – Old’ to ‘My Process’
  13. I was then able to delete the process template ‘My Process – Old’ as it has no associated team projects and was not the default

As I customise my primary organisation’s process templates I can repeat this process to keep the processes in sync between organisations.  Note that in future migrations I won’t have to do steps 2..6 as there are no manually created duplicated fields. So it should be more straight forward.

So a valid solution until any similar functionality is built into Azure DevOps, and there is no sign of that on the roadmap.

DPI problems after upgrading from Camtasia 8 to 2018

This is another of those posts I do so I don’t forget how I fixed something.

I have a requirement to record videos for a client in 720p resolution. As I use as SurfaceBook with a High-Res screen I have found the best way to do this is set my Windows screen resolution to 1280×720 and do all my recording at this as native resolution. Any attempt to record smaller portions of a screen or scale video in production have lead to quality problems, especially as remote desktops within remote desktops are required.

This has been working fine with Camtasia 8, but when I upgrade to Camtasia  2018.0.7 I got problems. The whole UI of the tool was unusable, it ignored the resizing/DPI changes.

The only fix I could find was to create a desktop shortcut to the EXE and set the Properties > Compatibility > Change high DPI settings > and check the ‘Override high DPI scaling behaviour’ and set this to ‘System’.

image

Even after doing this I still found the preview in the editing screen a little blurred, but usable. The final produced MP4s were OK.

Using Paths in PR Triggers on an Azure DevOps Pipelines Builds

When I started creating OSS extensions for Azure DevOps Pipelines (starting on TFSPreview, then VSO, then VSTS and now named Azure DevOps) I made the mistake of putting all my extensions in a single GitHub repo. I thought this would make life easier, I was wrong, it should have been a repo per extension.

I have considered splitting the GitHub repo, but as a number of people have forked it, over 100 at the last count, I did not want to start a chain of chaos for loads of people.

This initial choice has meant that until very recently I could not use the Pull Request triggers in Azure DevOps Pipelines against my GitHub repo. This was because all builds associated with the repo triggered on any extension PR. So, I had to trigger builds manually, providing the branch name by hand. A bit of a pain, and prone to error.

I am pleased to say that with the roll out of Sprint 140 we now get the option to add a path filter to PR triggers on builds linked to GitHub repo; something we have had for Azure DevOps hosted Git repos since Sprint 126.

So now my release process is improved. If I add a path filter as shown below, my build and hence release process trigger on a PR just as I need.

image

It is just a shame that the GitHub PR only checks the build, not the whole release, before saying all is OK. Hope we see linking to complete Azure DevOps Pipelines in the future.

Videos do not play in VSTS WIKI via relative links – workaround

The Problem

The documentation for the VSTS WIKI suggests you can embed a video in a VSTS WIKI using the markdown/HTML

<video src="_media/vstswiki_mid.mp4" width=400 controls>
</video>

Problem is that this does not seem to work, the MP4 just does not appear, you get an empty video player.

However, if you swap to a full URL it does work e.g.

<video src="https://sec.ch9.ms/ch9/7247/7c8ddc1a-348b-4ba9-ab61-51fded6e7247/vstswiki_high.mp4" width=400 controls> 
</video>

This is a problem if you wish to store media locally in your WIKI

The Workaround

The workaround is to either place the MP4 file in some URL accessible location e.g. some Azure web space (not really addressing the problem), or more usefully use the VSTS API to get the file out the repo that backs the WIKI.

The format of the HTML tag becomes

<video src="https://vstsinstance.visualstudio.com/MyTeamProject/_apis/git/repositories/MyTeamProject.wiki/Items?path=_media%2Fvstswiki_high.mp4 width=400" controls>
</video>

This will get the current version of the file on default branch, you can add extra parameters to to specify versions and branches if required as per the API documentation.

So not a perfect solution as you have to think about branches and versions, they are not handled automatically, but at least it does work

Registering an agent with VSTS and getting the message "Agent pool not found"

When you want to register a build agent with VSTS, you use the VSTS instance’s URL and a user’s Personal Access Token (PAT). Whilst doing this today I connected to the VSTS instance OK but got the error “Agent pool not found”.when I was asked to pick the agent pool to add the new agent to.

As the user who’s PAT I was using was a Build Administrator I was a bit confused, but then I remembered to check their user access level. It was set to Stakeholder, once this was changed to Basic I was able to register the agent without use.

Also, so as to not use up a Basic license, when I did not need to, I swapped the user back to being a Stakeholder once the agent was registered. This can be done as the token used for the actual build is not the one used to register it, but one assigned at build time by VSTS.

Hello sign in does not work on my first generation Surface Book after a rebuild – fixed

I have just rebuilt my first generation Surface Book from our company standard Windows image. These images are used by all our staff all the time without issue (on Surface and Lenovo devices), so I was not expecting any problems.

I used to rebuild my PC every 6 months or so, but got out the habit when I moved to a model I could not swap the hard drive out of as backup during the process (using a new disk for new install). I got around this by using Disk2VHD, not quite as good as I can’t just swap the disk back in, but I won’t have lost any stray files, even though I always aim to keep data in OneDrive, Source Control or SharePoint, so it should not been an issue anyway.

Anyway, the rebuild went fine, no issues until I tried to enable Hello to login using the camera. The process seemed to start OK, the wizard ran, but after a reboot there was no Hello login option.

After a bit of digging I found that in device manager there were no imaging devices at all – strange as the Camera App and Skype worked OK.

The Internet proved little help, suggesting the usual set of ‘you have a virus install our tool’ but after much more digging around I found that the cameras were all under ‘System Devices’ in device manager. So I then…

  1. Uninstalled them all (front, front IR and back)
  2. Scanned for hardware changes (they reappeared, still in ‘System Devices’
  3. Ran a Windows Update and some new Intel Drivers were downloaded
  4. I could then run the Hello setup wizard again, seems all old settings were lost

That was all much more complex than hoped for. My guess is that the system rebuild changed some firmware in a strange way that makes for the misdetection of the cameras.

Anyway it is working now.

Getting Remote Desktop Manager 2.7 working sanely with mixed high DPI screens

Updated 3 July 2018 – A colleague, Andy Davidson,  suggested mRemoteNG as an alternative tool to this address this issue. mRemoteNG also has the advantage that it support most major remoting technologies not just RDP, so I am giving that a try for a while.

This is one of those post I do mostly for myself so I don’t forget how I did something, it is all based on answers on SuperUser.Com, I can claim no credit

I have a SurfaceBook (first generation) and when I am in the office it is linked to an external monitor, with a different lower DPI, via a dock. If I use Remote Desktop (MSTSC) as built into Windows 10, I can drag sessions between the two monitors and the DPI shift is handled OK. However, if I use my preferred tool Remote Desktop Manager 2.7 (as it allow me to store all my commonly used RDP settings) I am in DPI hell. I either get huge fonts or microscopic ones. This is bad whether working on the single high DPI laptop screen work with an external screen.

As the SuperUser.Com post states the answer is to change the compatibility settings for the manager by right clicking on the file “C:\Program Files (x86)\Microsoft\Remote Desktop Connection Manager\RDCMan.exe”, selecting compatibility, change high DPI settings, and unchecking high DPI setting override

image

Once this was done, I have readable resolutions on all screens.

Why did I not do a better search months ago?

A workaround for the error ‘TF14061: The workspace ws_1_18;Project Collection Build Service does not exist’ when mapping a TFVC workspace

Whilst writing some training material for VSTS I hit a problem creating a TFVC workspace. I was using VS2017, linking a TFVC Repo to a local folder. I was connecting to the VSTS instance using an MSA.

In Team Explorer, when I came to do a ‘Map & Get’ to map the source locations I got a ‘TF14061: The workspace ws_1_18;Project Collection Build Service does not exist’ error

image

Strange error, which I could see no obvious reason for. Turns out the work around was just to press the ‘Advanced’ link/button and accept the defaults

Opps, I made that test VSTS extension public by mistake, what do I do now?

I recently, whilst changing a CI/CD release pipeline, updated what was previously a private version of a VSTS extension in the VSTS Marketplace with a version of the VSIX package set to be public.

Note, in my CI/CD process I have a private and public version of each extension (set of tasks), the former is used for functional testing within the CD process, the latter is the one everyone can see.

So, this meant I had two public versions of the same extension, confusing.

Turns out you can’t change a public extension back to be private, either via the UI or by uploading a corrected VSIX. Also you can’t delete any public extension that has ever been downloaded, and my previously private one had been downloaded once, by me for testing.

So my only option was to un-publish the previously private extension so only the correct version was visible in the public marketplace.

This meant I had to also alter my CI/CD process to change the extensionID of my private extension so I could publish a new private version of the extension.

Luckily, as all the GUIDs for the tasks within the extension did not change once I had installed the new version of the extension I had mispublished in my test VSTS instance my pipeline still worked.

Only downside is I am left with an un-publish ‘dead’ version listed in my private view of the marketplace. This is not a problem, just does not look ‘neat and tidy’