BM-Bloggers

The blogs of Black Marble staff

401.1 Permission error with on-premises TFS when accessing the API with a PAT

Background

If you are creating VSTS build extensions you will need to get the build or release’s PAT token if you wish to call the VSTS REST API.

This is done using a call like this (Node)

 

import tl = require('vsts-task-lib/task');

var auth = tl.getEndpointAuthorization('SYSTEMVSSCONNECTION', false);

if (auth.scheme === 'OAuth') {

var token = auth.parameters['AccessToken'];

 

or (PowerShell)

 

$vssEndPoint = Get-ServiceEndPoint -Name "SystemVssConnection" -Context $distributedTaskContext

$personalAccessToken = $vssEndpoint.Authorization.Parameters.AccessToken

 

You pop the resultant PAT into the headers of your REST web request and you are away and running.

The Problem

I hit a problem using this logic on VSTS Extension when they are run on TFS. On VSTS all was fine, but on TFS I got an unexpected 401.1 permission error on the first REST call i.e. I could not access the VSTS REST API

I tried setting fiddling with rights of my build user account, it was not that. Also I tried setting the ‘Allow scripts to access OAuth token’ property for the build/release agent

 

image

But this does not help either. This option just makes the PAT available as an environment variable, so you don’t need to use the code shown above.

And anyway – it was all worked on VSTS so it could not have been that!

Solution

The answer was I had basic authentication enabled on my Test TFS VM, as soon as this is disabled (the default) everything leapt into life.

image

Presenting at an event in Leeds - Making it easy to migrate your ALM process to the Cloud

Do you find your TFS server gets forgotten?

  • It is not owned by the IT department and the Development team don’t have the time to support it fully, it never gets patched or upgrades?
  • Or maybe you are adopting a cloud first strategy for all you systems?

Well maybe it is time to consider moving your on-premises TFS instance to VSTS?

On the 9th of May at the Crowne Plaza Hotel in Leeds I will be presenting at a Black Marble /Microsoft event where we will be looking at Microsoft’s new high fidelity VSTS database import tools that can be used to move a TFS instance to VSTS.

I will also be considering the pros and cons of the other migration options available to you. Hopefully making this a very useful session if you are considering a move to VSTS from TFS or any other source control ALM solution.

Hope to see you there, to register click here

 

image

Enumerating BizTalk 2016 Features for a Command-Line Installation

As with previous versions of BizTalk Server, you can perform the installation using the GUI or a command-line. To use the command-line installation, you’ll need the list of features that can be installed to add to the /AddLocal command-line. The available documentation for a silent installation of BizTalk Server at https://msdn.microsoft.com/en-us/library/jj248690.aspx relate to BizTalk Server 2013 and 2013 R2 (see https://msdn.microsoft.com/en-us/library/mt743078.aspx for ‘BizTalk Server 2016: What’s new, and installation’, then follow the link Appendix A: Silent installation near the bottom of the navigation menu at the left to get to the above page); there’s nothing that I’ve found so far that provides the setup.exe command line switches, or a list of features for use in a silent installation specifically for BizTalk Server 2016. Note that blindly following the previous guidance and using certain specific /AddLocal features results in an installation failure!

Getting hold of the command-line parameters for setup.exe is, of course, simple. Just run setup.exe with the ‘/?’ switch from a command prompt to get the following:

Command Description
/help or /? or /h Help and quick reference option.
/s <Configuration XML file> Silent Installation of features found in Configuration file.
/passive Passive Installation. Only progress bar will be displayed.
/norestart Supress restart.
/forcerestart Always restart after installation.
/promptrestart Prompts before restarting. This option cannot be used with the /quiet option.
/x or /uninstall Uninstalls the product.
/L <Logfile> Writes logging information into a logfile at the specified path. Always uses verbose MSI logging and appends to existing file.
/IGNOREDEPENDENCIES Bypass checks for downloadable prerequisites.
/INSTALLDIR <Install path> Specify the full path to product install location.
/COMPANYNAME <companyname> Sets the company name.
/USERNAME <User name> Sets the user name.
/ADDLOCAL ALL Install all features.
/REMOVE ALL Remove all features.
/REPAIR ALL Repair installation.
/CABPATH <cabfile> Specify a local path to a redistributable CAB file.
/CEIP Opt in to BizTalk Server Customer Experience Improvement Program.

These commands correspond to those listed on the silent installation page for BizTalk 2013 mentioned above with the exception that the final two commands listed on the web page appear to be missing from the above list generated by BizTalk Server 2016.

The /AddLocal command-line parameter details the features that will be installed. On the silent installation web page, there is a link to follow to the list of features (at http://go.microsoft.com/fwlink/p/?LinkID=189319), however if you browse to that page, you’ll notice that it is marked as the features for BizTalk Server 2010. There are issues using some of the parameters for the installation of BizTalk Server 2016, so it seemed worthwhile attempting to enumerate the parameters that are available to a BizTalk Server 2016 installation.

The installation MSI for BizTalk Server 2016 can be opened using Orca (Orca.exe is a database table editor for creating and editing Windows Installer packages and merge modules – see https://msdn.microsoft.com/en-us/library/windows/desktop/aa370557(v=vs.85).aspx for acquisition and installation instructions) to get the list of features. The screen shot below shows a partial view of the ‘Feature’ table from the ‘Microsoft BizTalk Server64.msi’ file:

Orca Features Table for BizTalk Server 2016 MSI

The information in the ‘Feature’ table, along with information gleaned by running the installer, ticking specific single components and then examining the setup log file can be reorganised to give the following:

Feature AddLocal Command
Portal Components BizTalk, WMI, InfoWorkerApps
      Business Activity Monitoring BAMPortal
Developer Tools and SDK BizTalk, WMI, AdapterImportWizard, BizTalkExplorer, BizTalkExtensions, DeploymentWizard, Designer, Development, Migration, MsEDIMigration, MsEDISchemaExtension, MsEDISDK, OrchestrationDesigner, PipelineDesigner, SDK, TrackingProfileEditor, VSTools, WCFDevTools, XMLTools
Documentation Documentation
Server Runtime BizTalk, WMI, Engine, MOT, MSMQ, Runtime
      BizTalk EDI/AS2 Runtime MsEDIAS2, MsEDIAS2StatusReporting
      Windows Communication Foundation Adapter WCFAdapter
Administration Tools and Monitoring BizTalk, WMI, AdminAndMonitoring, AdminTools, BAMTools, BizTalkAdminSnapIn, HealthActivityClient, MonitoringAndTracking, PAM
      Windows Communication Foundation Administration Tools WcfAdapterAdminTools
Additional Software BizTalk, WMI, AdditionalApps
      Enterprise Single Sign-On Administration Module SSOAdmin
      Enterprise Single Sign-On Master Secret Server SSOServer
      Business Rules Components RulesEngine
      MQSeries Agent MQSeriesAgent
      BAM Alert Provider OLAPNS
      BAM CLient FBAMCLIENT
      BAM-Eventing BAMEVENTAPI
      Project Build Component ProjectBuildComponent

Notes:

  • The ‘BizTalk’ and ‘WMI’ feature are specified in numerous places in the table above. You only need to specify each of these items once.
  • The parameters are case sensitive. Specifying a parameter incorrectly, e.g. OlapNs rather than OLAPNS will result in a silent installation failure.
  • When adding the parameters to the command line, it is important that there is no space between the items. Including a space (e.g. ‘BizTalk, WMI, AdditionalApps’ rather than ‘BizTalk,WMI,AdditionalApps’) will result in a silent installation failure.
  • One of the features, ‘SDKScenarios’, is never mentioned in the setup log file. It is assumed that this feature is automatically installed if required by the parent feature (SDK), however including it within the AddLocal command line parameter list doesn’t seem to cause any issues.

New Cross Platform version of my Generate Release Notes VSTS Extension

My Generate Release Notes VSTS extension has been my most popular by a long way. I have enhanced it, with the help of others via pull requests, but there have been two repeating common questions that have not been resolved

  1. Is it cross platform?
  2. Why does it show different work items and commit associations to the VSTS Release Status UI?

Well the answer to the first is that the core of the logic for the extension came from a PowerShell script we used internally, so PowerShell was the obvious first platform, especially as though my PowerShell skills are not great, my Node was weaker!

The second issue is due to my original extension and VSTS’s UI doing very different things. My old extension was based around inspecting build results, so when working in a release it finds all the builds between the current release and last successful one and looks at the details of each build in turn, building a big list or changes. VSTS’s Release summary UI does not do this, it make a few current undocumented ‘compare this to that’ API calls to get the lists.

In an attempt to address both these questions I have over the past few weeks created a new Cross Platform Generate Release Notes Extension. Now don’t worry, the old one is still there and supported, they do different jobs. This new extension is cross platform and tries to use the same API calls the VSTS Release summary UI uses.

There are of course a few gotchas

  • I did have to adopt a work around for TFVC changeset history as Microsoft use an old internal API call, but that that was the only place I had to do this. So apologies if there are any differences in the changesets returned.
  • The template format is very similar to that used in my original Generate Release Notes VSTS extension, but due to the change from PowerShell to Node I had to move from $($widetail.fields.'System.Title') style to ${widetail.fields['System.Title']}

So I hope people find this new extension useful, I can now go off happily closing old Issues in GitHub

VSTS Build Task Development - Now that is a misleading error message !

I have been working on a new Node.JS based VSTS Build Extension, so loads of Node learning going on as it is a language I have not done much with in the past. I expect to get caught out in places, but I have just wasted a good few hours on this one!

I am working in VS Code using Typescript to generate my Node based task. Whilst hacking around I have been just running the script in the VS Code debugger, getting the logic right before packaging it up and testing within a VSTS Build extension.

Everything was working until I did a big refactor, basically moving functions to a module. I suddenly started getting the following error trying to make a REST call

Exception occured

Error: Error: Hostname/IP doesn't match certificate's altnames: "Host: richardfennell.vsrm.visualstudio.com. is not in the cert's altnames: DNS:*.vsrm.visualstudio.com, DNS:vsrm.visualstudio.com, DNS:*.app.vsrm.visualstudio.com, DNS:app.vsrm.visualstudio.com"

image

 

I looked for ages to see how I had broken the REST call, all to no avail. In the end I rolled back and had to step through the refactor in small steps (smaller steps I should probably have taken anyway)

In the end I found the issue. The problem was in my early testing I had hard coded my input parameters e.g.

var templateFile = "template.md”;

Whilst stating to wire up the code as a VSTS Task I had started to swap in calls to the VSTS task Library

import tl = require('vsts-task-lib/task');

Correction – all the tl.xxx calls seem to cause a problem, avoid them for local testing

Now for items such as logging this works fine whether the logic is running in VS Code’s debugger or on a VSTS Build Agent, so I could use the following line in VS Code or on a VSTS Build Agent.

tl.Debug(“My Message”);

Where it does not work is for Task inputs. I had assume that

var templateFile = tl.getInput("templatefile");

Would return null/undefined when running in the VS Code debugger, but no, it causes that strange exception.

Once I removed the all the getInput calls my error went away.

Hope this save someone else some time

Migrating projects from CodePlex to GitHub due to CodePlex shutting down at the end of year

It has just been announced by Microsoft that it’s Open Source service CodePlex is shutting down before the end of the year. The reality is that the Microsoft focused Open Source community, and Microsoft itself, have moved to GitHub a good while ago.

I think I, like most developers, have moved any active Open Source projects to GitHub a good while ago,  but I still had legacy ones on CodePlex.

Microsoft have provided a nicely documented process to move the key assets of projects, whether TFVC or Git based, to GitHub. This process worked for me. However, I will suggest a could of changes/additions

  1. I would not export the WIKI docs as detailed in the process. I don’t want my old CodePlex Wiki pages in the new GitHub code repository as a folder. I think it is better to move each page over to a GitHub WIki. I only had few pages, so I did this by hand. I used this nice little tools  from Dom Christie to convert the CodePlex HTML based pages to Markdown which I cut and pasted into the new repo’s Wiki, fixing URLs as I went.
  2. I decided I needed to consider release downloads. The process does not do address this area. I thought I should bring over at least the last set of release binaries for my projects as a Github Releases. The reason was that the chances are for any old inactive project on CodePlex you won’t have the tools to hand to re-build the code easily, so just in case it is best to keep the last built version to hand as a release
  3. The process does not bring over Issues, but this was not a problem for me, the projects I have been superseded by active ones already on Github, so the issues are irrelevant

So if you have old CodePlex projects and you don’t want them to disappear think about moving them before the service is shutdown, you have until December 2017.