BM-Bloggers

The blogs of Black Marble staff

Restarting VS Code fixed NPM INSTALL intermittent EPERM issues

Whilst doing some NPM build work for VSTS Extensions I kept getting intermittent EPERM errors about renaming Windows files during NPM install (as discussed on GitHub)l. When you get this it completely blocks any development.

As the Github issue discusses there are many possible reasons for this issue, and many proposed potential solutions. However the only one that worked for me was to restart VS Code; as this appeared to be locking the node_modules folder somehow. This was even though I could delete it via Windows Explorer without any problems.

A quick restart of VS Code and all was good again for a while, good enough to work with.

Duplicate project GUID blocking SonarQube analysis of Windows 10 Universal Projects

I have working on getting a Windows 10 Universal application analysed with SonarQube 6.x as part of a VSTS build. The problem has been that when the VSTS task to complete the SonarQube analysis ran I kept getting an error in the form

 

WARNING: Duplicate project GUID: "8ace107e-8e3c-4a1b-9920-e76eb1db5e53". Check that the project is only being built for a single platform/configuration and that that the project guid is unique. The project will not be analyzed by SonarQube. Project file: E:\Build1\_work\58\s\BlackMarble.Victory.Common.Module.csproj

… plus loads more similar lines.
The exclude flag has been set so the project will not be analyzed by SonarQube. Project file: E:\Build1\_work\58\s\BlackMarble.Victory.Ux.Common.csproj
… plus loads more similar lines.

WARNING: Duplicate project GUID: "1e7b2f4e-6de2-40ab-bff9-a0c63db47ca2". Check that the project is only being built for a single platform/configuration and that that the project guid is unique. The project will not be analyzed by SonarQube. 2017-06-09T15:50:41.9993583Z ##[error]No analysable projects were found but some duplicate project IDs were found. Possible cause: you are building multiple configurations (e.g. DEBUG|x86 and RELEASE|x64) at the same time, which is not supported by the SonarQube integration. Please build and analyse each configuration individually.
Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.

Turns out the issue was that even though my CI build was only set to create an x86|Debug build the act of creating the .APPX package was causing both x64 and ARM builds to be build too, this was too much for SonarQube as it though I had a multiplatform build..

The answer was to pass a parameter into the Visual Studio build task to disable the creation of the .APPX package.

The parameter override required is /p:AppxBundle=Never. This overrides the setting of Always that was set in the .CSProj file.

 

image

Once this change was done analysis completed as expected. Just need to fix all the issues it found now!

Debugging Typescript in Visual Studio Code

This is one of those posts I do as a reminder to myself. I have struggled to get debugging working in VSCode for Typescript files. If I set breakpoints in the underlying generated JavaScript they worked, but did not work if they were set in the Typescript source file. There are loads of walkthroughs and answers on Stackoverflow, but all with that vital little bit (for me) missing. So this is what I needed to do for my usage scenario…

Whilst developing a Node based VSTS extension I have the following structure

Git repo root (folder)

--- .vscode (folder)

------ launch.json

--- mystuff.src   (the source .TS files)

------ script.ts

------ tsconfig.json

--- mystuff    (the target folder for the .JS files)

I develop my Typescript in the .src folder and use the TSC compiler to generate the .JS file into the target folder, running the ‘tsc –p .’ command whilst sitting in the .src folder.

Note: I actually run this Tsc command using Gulp, but this post does not need to go into detail of that.

The key thing is to make sure the two .json files have the correct options

The tsconfig.json is

{

"compilerOptions": {

"target": "ES6",

"module": "commonjs",

"watch": false,

"outDir": "../mystuff/",

"sourceMap": true

},

"exclude": [

"node_modules"

]

}

The important lines are highlighted

  • The path to generate to .JS to – for me it was important to generate to a different folder as this made it easier to create the VSTS extension packages without shipping the .TS files by accident.This was part of my debugging problem as if the .ts and .js files are in the folder the should be no issues
  • Creating the source map which enables debugging, the .JS.MAP files

The launch.json is

{

"version": "0.2.0",

"configurations":

[

{

"type": "node",

"request": "launch",

"name": "Node – my stuff",

"program": "${workspaceRoot}/mystuff.src/script.ts",

"outFiles": ["${workspaceRoot}/mystuff/*.js"]

}

]

}

The critical lines, and the ones I messed up are

  • Program must point at the .TS file
  • Outfiles must point to the location of the .JS and .JS.Map files in the target folder and those square brackets [] are vital

Once I had all this in place I could set breakpoints the .TS file and they worked

401.1 Permission error with on-premises TFS when accessing the API with a PAT

Background

If you are creating VSTS build extensions you will need to get the build or release’s PAT token if you wish to call the VSTS REST API.

This is done using a call like this (Node)

 

import tl = require('vsts-task-lib/task');

var auth = tl.getEndpointAuthorization('SYSTEMVSSCONNECTION', false);

if (auth.scheme === 'OAuth') {

var token = auth.parameters['AccessToken'];

 

or (PowerShell)

 

$vssEndPoint = Get-ServiceEndPoint -Name "SystemVssConnection" -Context $distributedTaskContext

$personalAccessToken = $vssEndpoint.Authorization.Parameters.AccessToken

 

You pop the resultant PAT into the headers of your REST web request and you are away and running.

The Problem

I hit a problem using this logic on VSTS Extension when they are run on TFS. On VSTS all was fine, but on TFS I got an unexpected 401.1 permission error on the first REST call i.e. I could not access the VSTS REST API

I tried setting fiddling with rights of my build user account, it was not that. Also I tried setting the ‘Allow scripts to access OAuth token’ property for the build/release agent

 

image

But this does not help either. This option just makes the PAT available as an environment variable, so you don’t need to use the code shown above.

And anyway – it was all worked on VSTS so it could not have been that!

Solution

The answer was I had basic authentication enabled on my Test TFS VM, as soon as this is disabled (the default) everything leapt into life.

image

VSTS Build Task Development - Now that is a misleading error message !

I have been working on a new Node.JS based VSTS Build Extension, so loads of Node learning going on as it is a language I have not done much with in the past. I expect to get caught out in places, but I have just wasted a good few hours on this one!

I am working in VS Code using Typescript to generate my Node based task. Whilst hacking around I have been just running the script in the VS Code debugger, getting the logic right before packaging it up and testing within a VSTS Build extension.

Everything was working until I did a big refactor, basically moving functions to a module. I suddenly started getting the following error trying to make a REST call

Exception occured

Error: Error: Hostname/IP doesn't match certificate's altnames: "Host: richardfennell.vsrm.visualstudio.com. is not in the cert's altnames: DNS:*.vsrm.visualstudio.com, DNS:vsrm.visualstudio.com, DNS:*.app.vsrm.visualstudio.com, DNS:app.vsrm.visualstudio.com"

image

 

I looked for ages to see how I had broken the REST call, all to no avail. In the end I rolled back and had to step through the refactor in small steps (smaller steps I should probably have taken anyway)

In the end I found the issue. The problem was in my early testing I had hard coded my input parameters e.g.

var templateFile = "template.md”;

Whilst stating to wire up the code as a VSTS Task I had started to swap in calls to the VSTS task Library

import tl = require('vsts-task-lib/task');

Correction – all the tl.xxx calls seem to cause a problem, avoid them for local testing

Now for items such as logging this works fine whether the logic is running in VS Code’s debugger or on a VSTS Build Agent, so I could use the following line in VS Code or on a VSTS Build Agent.

tl.Debug(“My Message”);

Where it does not work is for Task inputs. I had assume that

var templateFile = tl.getInput("templatefile");

Would return null/undefined when running in the VS Code debugger, but no, it causes that strange exception.

Once I removed the all the getInput calls my error went away.

Hope this save someone else some time

New version of my Parameters.Xml Generator Visual Studio add-in now supports VS2017 too

I have just published Version 1.5 of my Parameters.Xml Generator Visual Studio add-in . After much fiddling this VSIX now supports VS2017 as well as VS2013 and VS2015.

The complexity was that VS2017 uses a new VSIX format, V3. You have to makes changes to the project that generates the VSIX and to the VSIX manifest too. The FAQ says you can do this within VS2015 my hand, but I had no luck getting it right. The recommended option, and the method I used, is to upgrade your solution to VS2017 (or the RC at the time of writing as the product has not RTM’d yet).

This upgrade process is a one way migration, and you do have to check/edit some key items

  • Get all your references right, as I was using the RC of VS2017  this meant enabling the use of Preview packages from Nuget in the solution.
  • Makes sure the install targets (of Visual Studio) match what you want to install too
  • Add prerequisites (this is the big new addition in the VSIX 3 format)
  • And the one that stalled me for ages – Make sure you reference the right version of the Microsoft.VisualStudio.Shell.<VERSION>.dll . You need to pick the one for the oldest version of Visual Studio you wish to target. In my case this was Microsoft.VisualStudio.Shell.12.0.dll. For some reason during the migration this got changed to Microsoft.VisualStudio.Shell.14.0.dll which gave the strange effect that the VSIX installed on 2013, 2015 and 2017. But in 2013, though I could see menu item, it did not work. This was fixed by referencing the 12.0 DLL .

Can’t add users to a VSTS instance backed by an Azure Directory

I have a VSTS instance that is backed by an Azure Directory. This is a great way to help secure a VSTS instance, only users in the Azure Directory can be added to VSTS, not just any old MSA (LiveIDs). This is a directory that can be shared with any other Azure based services such as O365, and centrally managed and linked to an on-premises Active Directory.

When I tried to add a user to VSTS, one that was a valid user in the Azure Directory, their account did not appear in the available users drop down.

 image

Turns out the problem was who I was logged in as. As yo can see from the screenshot I have three Richard accounts in the VSTS instance (and Azure Directory), a couple of MSAs and a guest work account from another Azure Directory. I was logged in as the guest work account.

All three IDs as administrators in VSTS, but it turned out I needed to be logged in as the MSA that owned the Azure subscription contains the Azure Directory. As soon as I used this account the dropdown populated as expected and I could add the users from the Azure Diretcory

image

Transform tool for transferring TFS 2015.3 Release Templates to VSTS

If you are moving from on-premises TFS to VSTS you might hit the same problem I have just have. The structure of a VSTS releases is changing, there is now the concept of multiple ‘Deployment Steps’ in an environment. This means you can use a number of different agents for a single environment – a good thing.

The downside this that if you export a TFS2015.3 release process and try to import it to VSTS it will fail saying the JSON format is incorrect.

Of course you can get around this with some copy typing, but I am lazy, so….

I have written a quick transform tool that converts the basic structure of the JSON to the new format. You can see the code as Github Gist

It is a command line tool, usage is as follows

  1. In VSTS create a new empty release, and save it
  2. Use the drop down menu on the newly saved release in the release explorer and export the file. This is the template for the new format e.g. template.json
  3. On your old TFS system export the release process in the same way to get your source file e.g. source.json
  4. Run the command line tool providing the name of the template, source and output file

    RMTransform template.json source.json output.json
  5. On VSTS import the newly create JSON file release file.
  6. A release process should be created, but it won’t be possible to save it until you have fixed a few things that are not transferred
    1. Associated each Deployment step with Agent Pool
    2. Set the user accounts who will do the pre-and post approvals
    3. Any secret variable will need to be reentered
      IMPORTANT - Make sure you save the imported process as soon as you can (i.e. straight after fixing anything that is stopping it being saved). If you don't save and start clicking into artifacts or global variable it seems to loose everything and you need to re-import

image

It is not perfect, you might find other issues that need fixing, but it save a load of copy typing

Deleting unwanted orphan XAML Build Controllers on a migrated VSTS instance

Whilst working with the VSTS Data Import Service I ended up migrating a TFS TPC up to VSTS that had an old XAML Build Controller defined. I did not need this XAML build controller, in fact I needed to remove it because it was using my free private build controller slot. Problem was I could not find a way to remove it via the VSTS (or Visual Studio Team Explorer) UI, and the VM that had been running the build controller was long gone.

The way I got rid of it in the end was the TFS C# API and a quick command line tool as shown below.

Note that you will need to delete any queued builds on the controller before you can delete it. You can do this via the VSTS browser interface.

Problem adding external AAD user to a directory backed VSTS instance

Background

I recently decided to change one of my VSTS instance to be directory backed. What this means is that in the past users logged in using LiveIDs (MSAs by their new name); once the VSTS instance was linked to an Azure Active Directory (AAD), via the Azure portal, they could login only if

  • they were using an account in the AAD
  • their MSA was listed as a guest in the AAD
  • they used a work ID in another AAD that is listed as a guest in my AAD
  • Thus giving me centralised user management.

    So I made the changes required, and the first two types of user were fine, but I had a problem with the third case. When I did the following

  • Added and external Work ID to my AAD directory (via the old management portal https://manage.windowsazure.com)
  • Added the user in my VSTS instance as a user
  • Granted the new user rights to access team projects.
  • All seemed to go OK, but when I tried to login as the user I got the error

    TF400813: The user 'db0990ce-80ce-44fc-bac9-ff2cce4720af\fez_blackmarble.com#EXT#@richardblackmarbleco.onmicrosoft.com' is not authorized to access this resource.

    clip_image002

    Solution

    With some help from Microsoft I got this fixed, seem to be an issue with Azure AD. The fix was to do the following

    1. Remove the user from VSTS account
    2. Go to the new Azure Portal (https://portal.azure.com/) and remove this user from the AAD
    3. Then re-add them as an external user back into the AAD (an invite email is sent)
    4. Add the user again to VSTS (another invite email is sent)
    5. Grant the user rights to the required team projects

    and this fixed the access problems for me. The key item for me I think was to use the new Azure portal.

    Hope it saves you some time