BM-Bloggers

The blogs of Black Marble staff

My TFSAlertsDSL project has moved to GitHub and become VSTSServiceHookDsl

Introduction

A while ago I create the TFSAlertsDSL project to provide a means to script responses to TFS Alert SOAP messages using Python. The SOAP Alert technology has been overtaken by time with the move to Service Hooks.

So I have taken the time to move this project over to the newer technology, which is supported both on TFS 2015 (onwards) and VSTS. I also took the chance to move from CodePlex to GitHub and renamed the project to VSTSServiceHookDsl.

Note: If you need the older SOAP alert based model stick with the project on CodePlex, I don’t intend to update it, but all the source is there if you need it.

What I learnt in the migration

Supporting WCF and Service Hooks

I had intended to keep support for both SOAP Alerts and Service Hooks in the new project, but I quickly realised there was little point. You cannot even register SOAP based alerts via the UI anymore and it added a lot of complexity. So I decided to remove all the WCF SOAP handling.

C# or REST TFS API

The SOAP Alert version used the older TFS C# API, hence you had to distribute these DLLs with the web site. Whilst factoring I decided to swap all the TFS calls to using the new REST API. This provided a couple of advantages

  • I did not need to distribute the TFS DLLs
  • Many of the newer function of VSTS/TFS are only available via the REST API

    Exposing JObjects to Python

    I revised the way that TFS data is handed in the Python Scripts. In the past I hand crafted data transfer objects for consumption within the Python scripts. The problem with this way of working is that it cannot handle custom objects, customised work items are a particular issue. You don’t know their shape.

    I found the best solution was to just return the Newtonsoft JObjects that I got from the C# based REST calls. These are easily consumed in Python in the general form

    workitem["fields"]["System.State"] 


    Downside is that this change does mean that any scripts you had created for the old SOAP Alert version will need a bit of work when you transfer to the new Service Hook version.

    Create a release pipeline

    As per all good projects, I created a release pipeline for my internal test deployment. My process was as follows

    • A VSTS build that builds the code from Github this
      • Complies the code
      • Run all the unit test
      • Packages as an MSDeploy Package
    • Followed by a VSTS release that
      • Sets the web.config entries
      • Deploys the MSDeploy package to Azure
      • Then uses FTP to uploaded DSL DLL to Azure as it is not part of the package

    image 

    Future Steps

    Add support for more triggers

    At the moment the Service Hook project supports the same trigger events as the old SOAP project, with the addition of support Git Push triggers.

    I need to add in the handlers for all the older support triggers in VSTS/TFS, specifically the release related ones. I suspect these might be useful.

    Create an ARM template

    At the moment the deployment relies on the user creating the web site. It would be good to add an Azure Resource Management (ARM) Template to allow this site to be created automatically as part of the release process

    Summary

    So we have a nice new Python and Service Hook based framework to help manage your responses to Service Hook triggers for TFS and VSTS.

    If you think it might be useful to you why not have a look at https://github.com/rfennell/VSTSServiceHookDsl.

    Interested to hear your feedback  

  • Transform tool for transferring TFS 2015.3 Release Templates to VSTS

    If you are moving from on-premises TFS to VSTS you might hit the same problem I have just have. The structure of a VSTS releases is changing, there is now the concept of multiple ‘Deployment Steps’ in an environment. This means you can use a number of different agents for a single environment – a good thing.

    The downside this that if you export a TFS2015.3 release process and try to import it to VSTS it will fail saying the JSON format is incorrect.

    Of course you can get around this with some copy typing, but I am lazy, so….

    I have written a quick transform tool that converts the basic structure of the JSON to the new format. You can see the code as Github Gist

    It is a command line tool, usage is as follows

    1. In VSTS create a new empty release, and save it
    2. Use the drop down menu on the newly saved release in the release explorer and export the file. This is the template for the new format e.g. template.json
    3. On your old TFS system export the release process in the same way to get your source file e.g. source.json
    4. Run the command line tool providing the name of the template, source and output file

      RMTransform template.json source.json output.json
    5. On VSTS import the newly create JSON file release file.
    6. A release process should be created, but it won’t be possible to save it until you have fixed a few things that are not transferred
      1. Associated each Deployment step with Agent Pool
      2. Set the user accounts who will do the pre-and post approvals
      3. Any secret variable will need to be reentered
        IMPORTANT - Make sure you save the imported process as soon as you can (i.e. straight after fixing anything that is stopping it being saved). If you don't save and start clicking into artifacts or global variable it seems to loose everything and you need to re-import

    image

    It is not perfect, you might find other issues that need fixing, but it save a load of copy typing

    Deleting unwanted orphan XAML Build Controllers on a migrated VSTS instance

    Whilst working with the VSTS Data Import Service I ended up migrating a TFS TPC up to VSTS that had an old XAML Build Controller defined. I did not need this XAML build controller, in fact I needed to remove it because it was using my free private build controller slot. Problem was I could not find a way to remove it via the VSTS (or Visual Studio Team Explorer) UI, and the VM that had been running the build controller was long gone.

    The way I got rid of it in the end was the TFS C# API and a quick command line tool as shown below.

    Note that you will need to delete any queued builds on the controller before you can delete it. You can do this via the VSTS browser interface.