But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

vNext Build editor filePath control always returns a path even if you did not set a value

You can use the filePath type in a vNext VSTS/TFS task as shown below 

{
     "name": "settingsFile",
     "type": "filePath",
     "label": "Settings File",
     "defaultValue": "",
     "required": false,
     "helpMarkDown": "Path to single settings files to use (as opposed to files in project folders)",
     "groupName":"advanced"
   }

to present a file picker dialog in the build editor that allows the build editor to pick a file or folder in the build’s source repository

image

While doing some task development recently I found that this control did not behave as I had expected

  • If a value is explicitally set then the full local path to selected file or folder (on the build agent) is returned e.g. c:\agent\_work\3\s\yourfolder\yourfile.txt – just as expected
  • If you do not set a value, or set a value then remove your setting when you edit a build, then you don’t get an empty string, as I had expected. You get the path to the BUILD_SOURCESDIRECTORY e.g. c:\agent\_work\3\s – makes sense when you think about it.

So, if as in my case, you wanted to have specific behaviour only when this values was set to something other than the repo root you need to add some guard code


if ($settingsFile -eq $Env:BUILD_SOURCESDIRECTORY )
{
    $settingsFile = ""
}

Once I did this my task behaved as a needed, only running the code when the user had set an explicit value for the settings file.

Fixing cannot load dashboard issues on BlogEngine.NET using sub blog aggregation

As I discovered during my BlogEngine upgrade, there is an effort within the project team to focus the codebase on three possible usage models on any given BlogEngine server instance:

  • Single blog with a user – a personal blog (default)
  • Single blog with many users – a team/company blog
  • Many blogs each with a single user – a set of related blogs that can be agregated togther

I needed the third option, problem was in its history our blog has been both of the other two types, so I have multiple user accounts for each blogs, and login usernames are repeated between individual blogs on the server.

This is not fundamentally an issue for a server running in the third mode, except on the primary blog that is setup to provide agregation of all the other blogs.  Even here, on a day to day basis, it is not an issue either, basic post RSS aggregation is fine. However, when you login as an administration user and try to access the dashboard you get the error

Item has already been added. Key in dictionary: 'displayname' Key being added: 'displayname'

The workaround I have used in the past was to temporarily switch off blog aggregation whenever I needed to access the primary blog dashboard – not the best solution.

After a bit of investigation of the codebase I found that this issue is due to the fact we had users  called ‘admin’ on the primary and all the child blogs. The fix I used was a bit of SQL to do some user renaming from ‘admin’ to ‘adminblogname’ . I needed to rename the username in a few tables.

AS USUAL BEWARE THIS SQL, MAKE SURE YOU HAVE A BACKUP BEFORE YOU USE IT, IT WORKS FOR ME BUT I MIGHT HAVE MISSED SOMETHING YOU NEED


update p
set p.SettingValue = concat (p.SettingValue , ' ', b.BlogName)
from be_Profiles p
    inner join be_Blogs b on
        b.BlogID = p.BlogId
where
SettingName ='displayname' and
SettingValue = 'admin';

update p
set p.UserName = concat (p.UserName , b.BlogName)
from be_Profiles p
    inner join be_Blogs b on
        b.BlogID = p.BlogId
where
username= 'admin';

update u
set u.UserName = concat (u.UserName , b.BlogName)
from be_Users u
    inner join be_Blogs b on
        b.BlogID = u.BlogId
where
username = 'admin';

update r
set r.UserName = concat (r.UserName , b.BlogName)
from be_UserRoles r
    inner join be_Blogs b on
        b.BlogID = r.BlogId
where
username = 'admin';

 

This is not a problem specific to admin users, any username duplication will cause the same error. This basic SQL script can be modified to fix any other user accounts you might have username clashes on.

Once this SQL was run I was able to login to the dashboard on the primary blog as expected.

Upgraded to BlogEngine.NET 3.2

I have just completed the upgrade of this blog server to the new release 3.2 of BlogEngine.NET. I did a manual upgrade (as opposed to the automated built in upgrade) as I needed to make a few changes from the default settings. The process I used followed the upgrade process document

  1. Downloaded the latest release and unzip the folder
  2. Run the SQL upgrade script (in /setup/sqlserver folder), this adds some new DB constraints
  3. Created a IIS web site using the new release
  4. Copied in the sample web.config from the /setup/sqlserver folder.
  5. Copied in my App_DATA folder
  6. Accessed my site

As I had not copied anything from the old custom folder, I had theme issues at this point. However, I decided to moved all the blogs to the newest generation of theme templates, so did a quick fix up by hand on each one, picking the required theme and making sure any settings, like Twitter accounts, were set (note these are set on a per blog/per theme basis, so changing a theme means you need to reenter any custom values). I also needed to copy in a few missing logos and any extra widgets from my old custom folder the blogs were using.

Once this was all done I had an upgraded blog server.

Running CodeUI tests on a VM with on remote desktop session open as part of a vNext build

If you want to run CodeUI tests as part of a build you need to make sure the device running the test has access to the UI, for remote VMs this means having a logged in session open and the build/test agent running interactivally. Problem is what happens when you disconnect the session. UNless you manage it you will get the error

Automation engine is unable to playback the test because it is not able to interact with the desktop. This could happen if the computer running the test is locked or it’s remote session window is minimized

In the past I would use a standard TFS Lab Management Environment to manage this,you just check a box to say the VM/PC is running coded UI tests and it sorts out the rest. However, with the advent of vNext build and the move away from Lab Manager this seems overly complex.

It is not a perfect solution but this works

  1. Make sure the VM autologs in and starts your build/test agents in interactive mode (I used SysInternal AutoLogin to set this up)
  2. I connect to the session and make sure all is OK, but I then disconnect redirecting the session
    • To get my session ID, at the command prompt, I use the command query user
    • I then redirect the session tscon.exe RDP-Tcp#99 /dest:console, where RDP-Tcp#99 is my session ID
  3. Once I was disconnected my CodeUI test still run

I am sure I can get a slicker way to do this, but it does fix the immediate issue

Updated:

This bit of Powershell code could be put in a shortcut on the desktop to do the job, you will want to run the script as administrator

$OutputVariable = (query user) | Out-String

$session = $OutputVariable.Substring($OutputVariable.IndexOf("rdp-tcp#")).Split(" ")[0]

& tscon.exe $session /dest:console

Nuget restore fails to restore all the files on VSTS build if using project.json files

We are currently working on updating a Windows 8 application to be a Windows 10 Universal application. This has caused a few problem on a TFS vNext automated build box. The revised solution builds fine of the developers box and fine on the build VM if opened in Visual Studio, but fails if built via the VSTS vNext build CI MSBuild process showing loads of references missing.

Turns out the issue was due to Nuget versions.

The problem was that as part of the upgrade the solution had gained some new projects. These used the new project.json file to manage their Nuget references, as opposed to the old packages.config file. Visual Studio 2015 handles these OK, hence the build always working in the IDE, but you need Nuget.exe 3.0 or later for it to handle the new format. The version of Nuget installed as part of my vNext build agent was 2.8. So no wonder it had a problem.

To test my assumptions I added a Nuget Installer task to my build and set an explicit path to the newest version of Nuget.

image

Once this was done my build was fine.

So my solution options are

  1. Don’t use the agents shipped with TFS 2015, get newer ones fro VSTS this has a current version of Nuget (just make sure your agent/server combination is supported. I have had issues with the latest VSTS agent and a TFS 2015 RTM instance)
  2. Manually replace the version of Nuget.exe in my build agent tools folder – easy to forget you did but works
  3. Place a copy of the version of Nuget.exe I want on each build VM and reference it s path explicitly (as I did to diagnose the problem)

The first option is the best choice as it is always a good plan to keep build agents up to date

An out-the-box way to let local Hyper-V VMs see the Internet without using a DD-WRT router

I have posted in the past about using a DD-WRT virtual router to bridge between local VMs on my development PC and the outside world

 

 

With the recent changes in HyperV on Windows 10 and Server 2016 we have an alternative as discussed by Thomas Maurer in his post. You can use a NATSwitch, thus removing the need for router VM. This does however raise different issues, that of address assignment, the router was also a DHCP server. However, it does mean I don’t have to mess around with manually setting external IP addresses for the router each time I join a different WIFI network. So on the whole I think iti is a better solution.

My new setup is as follows.

image

 

  • Using Powershell, create the NATSwitch as per Thomas Maurer’s post.
  • On each guest VM set a fixed IP address e.g. 172.92.91.2, with a default route of the NATSwitch’s port on the host OS i.e. 172.92.91.1 and a publicly accessible DNS such as a Google’s 8.8.8.8. Once setup it should be possible to access the internet from the guest VM
  • To access each guest VM from the host OS you can just use it’s IP address e.g. 172.92.91.2, this works because the host OS has a connection on the same NATSwitch network, 172.92.91.1. It makes sense to add a hosts files entry on the Windows 10 PC so that the user has a friendly name to access each guest VM e.g.

                    172.91.92.2        typhoontfs

Once this is all done you seem to have a workable system, only time will tell how it works in practice.

Upgrading to SonarQube 5.2 in the land of Windows, MSBuild and TFS

SonarQube released version 5.2 a couple of weeks ago. This enabled some new features that really help if you are working with MSbuild or just on a Windows platform in general. These are detailed in the posts

The new ability to manage users with LDAP is good, but one of the most important for me is the way 5.2 ease the configuration with SQL in integrated security mode. This is mentioned in the upgrade notes; basically it boils down to the fact you get better JDBC drivers with better support for SQL Clustering and security.

We found the upgrade process mostly straight forward

  1. Download SonarQube 5.2 and unzipped it
  2. Replaced the files on your SonarQube server
  3. Edited the sonar.properties file with the correct SQL connection details for an integrated security SQL connection. As we wanted to move to integrated security we did not need to set the sonar.jdbc.username setting.
    Important: One thing was not that clear, if you want to use integrated security you do need the sqljdbc_auth.dll file in a folder on a search path (C:\windows\system32 is an obvious place to keep it). You can find this file on MSDN
  4. Once the server was restarted we ran the http://localhost:9000/setup command and it upgraded our DBs

And that was it, for the upgrade. We could then use the standard SonarQube upgrade features to upgrade our plug-ins and to add the new ones like the  LDAP one.

Once the LDAP plug-in was in place (and the server restarted) we were automatically logged into SonarQube with our Windows AD accounts, so that was easy.

However we hit a problem with the new SonarQube 5.2 architecture and LDAP. The issue was that with 5.2 there is now no requirement for the upgraded 1.0.2 SonarQube MSBuild runner to talk directly to the SonarQube DB, all communication is via the SonarQube server. Obviously the user account that makes the call to the SonarQube server needs to be granted suitable rights. That is fairly obvious, the point we  tripped up on was ‘who was the runner running as?’ I had assumed it was as the build agent account, but this was not the case. As the connection to SonarQube is a TFS managed service, it had its own security credentials. Prior to 5.2 these credentials (other than the SonarQube server URL) had not mattered as the SonarQube runner made it own direct connection to the SonarQube DB. Post 5.2, with no DB connection and LDAP in use,  these service credentials become important. Once we had set these correctly, to a user with suitable rights, we were able to do new SonarQube analysis runs.

image

One other item of note. The change in architecture with 5.2 means that more work is being done on the server as opposed the client runner. The net effect of this is there is a short delay between runs being completed and the results appearing on the dashboard. Once expect it, it is not an issue, but a worry the first time.

Why you need to use vNext build tasks to share scripts between builds

Whilst doing a vNext build from a TFVC repository I needed map both my production code branch and a common folder of scripts that I intended to use in a number of builds, so my build workspace was set to

  • Map – $/BM/mycode/main                                     - my production code
  • Map – $/BM/BuildDefinations/vNextScripts  - my shared PowerShell I wish to run in different builds e.g. assembly versioning.

As I wanted this to be a CI build, I also  set the trigger to $/tp1/mycode/main

The problem I found was that with my workspace set as above, the associated changes for the build include anything checked into $/BM and below. Also the source branch was set as $/BM

image

 

To fix this problem I had to remove the mapping to the scripts folder, once this was done  the associated changes shown were only those for my production code area, and the source branch was listed correctly.

But what to do about running my script?

I don’t really want to have to copy a common script to each build, huge potential for error there, and versioning issues if I want a common script in all build. The best solution I found was to take the PowerShell script, in my case the sample assembly versioning script provided for VSO, and package it as a vNext build Task. It took no modification just required the addition of a manifest file. You can find the task on my vNextBuild Github repo

This custom task could then be uploaded to my TFS server and used in all my builds. As it picks up it variable from environment variables it required so configuration, extracting the version number for the build number format.

image

If you wish to use this task, you need to follow the same instructions to setup your development environment as for the VSO Tasks, then

  1. Clone the repo https://github.com/rfennell/vNextBuild.git
  2. In the root of the repo run gulp to build the task
  3. Use tfx to upload the task to your TFS instance

Why can’t I assign a VSO user as having ‘eligible MSDN’ using an AAD work account?

When access VSO you have two authentication options; either a LiveID (or an MSA using it’s newest name) or a Work Account ID (a domain account). The latter is used to provide extra security, so a domain admin can easily control who has access to a whole set of systems. It does assume you have used Azure Active Directory (AAD) that is sync’d with your on premises AD, and that this AAD is used to back your VSO instance. See my previous post on this subject.

If you are doing this the issue you often see is that VSO does not pickup your MSDN subscription because it is linked to an MSA not a work account. This is all solvable, but there are hoops to jump through, more than there should be sometimes.

Basic Process

First you need to link your MSDN account to a Work Account

  • Login to https://msdn.micrsoft.com with the MSA that is associated with your MSDN account.
  • Click on the MSDN subscriptions menu option.
  • Click on the Link to work account  and enter your work ID. Note that it will also set your Microsoft Azure linked work account

 

image


Assuming your work account is listed in your AD/AAD, over in VSO you should now be able to …

  • Login as the VSO administrator
  • Invite any user in the AAD to your VSO instance via the link https://[theaccount].visualstudio.com/_user . A user can be invited as
    • Basic – you get 5 for free
    • Stakeholder – what we fall back to if there is an issue
    • MSDN Subscription – the one we want (in screenshot below the green box shows a user where MSDN has been validated, the red box is a user who has not logged in yet with an account associated with a valid MSDN subscription)

image

  • Once invited a user gets an email so they can login as shown below. Make sure you pick the work account login link (lower left. Note that this is mocked up in the screen shot below as which login options are shown appears in a context sensitive way, only being shown the first time a user connects and if the VSO is AAD backed. If you pick the main login fields (the wrong ones) it will try to login assuming the ID is an MSA, which will not work. This is particularly a confusing issue if you used the same email address for your MSA as your Work Account, more on this in the troubleshooting section

 image

  • On later connections only the work ID login will be shown
  • Once a user has logged in for the first time with the correct ID, the VSO admin should be able to see the MSDN subscription is validated

Troubleshooting

We have seen problem that though the user is in the domain and correctly added to VSO it will not register that the MSDN subscription is active. These steps can help.

  • Make sure in the  https://msdn.microsoft.com portal you have actually linked your work ID. You still need to explicably do this even if your MSA and Work ID use the same email address e.g.   user@domain.com. Using the same email address for both IDs can get confusing, so I would recommend considering you setup your MSA email addresses to not clash with your work ID.
  • When you login to VSO MAKE SURE YOU USE THE WORK ID LOGIN LINK (LHS OF DIALOG UNDER VSO LOGO) TO LOGIN WITH A WORK ID AND NOT THE MAIN LIVEID FIELDS. I can’t stress this enough, especially if you use the same email address  for both the MSA and work account
  • If you still get issues with picking up the MSDN subscription
    • In VSO the admin should set the user to be a basic user
    • In  https://msdn.microsoft.com the user should make sure they did not make any typo's when linking the work account ID
    • The user should sign out of VSO and back in using their work ID, MAKE SURE THEYUSE THE CORRECT WORK ID LOGIN DIALOG. They should see the features available to a basic user
    • The VSO admin should change the role assignment in VSO to be MSDN eligible and it should flip over without a problem. There seems to be no need to logout and back in again.

Note if you assign a new MSA to an MSDN subscription it can take a little while to propagate, if you get issues that activation emails don’t arrive, pause a while and try again later. You can’t do any of this until your can login to MSDN with your MSA.

Release Manager - New deployment is not allowed as an another deployment is in progress

Whilst working with a vNext Release Management pipeline I started seeing the error

Microsoft.TeamFoundation.Release.Common.Helpers.OperationFailedException:
New deployment is not allowed as an another deployment is in progress.
Retry the deployment after sometime.

Problem was I could not see any blocked or paused deployment releases. All Internet searches mentioned multiple pipelines that share components, but this was not the issue.

Eventually I found the issue, my release pipeline included a step that ran CodedUI tests via TCM, hence a previous running of this template had triggered the test via TCM, but they had stalled. I found this by looking in MTM.

image

Release Management was just saying the release was rejected with the above error message, no clue about the unfinished test run. Not that helpful.

You might have expect Release Management to return only after the test had timed out, but that is only down to whether you set the release pipeline to wait or not, I had set mine not to wait.

Once I stopped this test run via MTM all was OK.