Don’t skimp on resources for GHES for demo instances

I wanted to have a look at some GitHub Enterprise Server (GHES) upgrade scenarios so decided to create a quick GHES install on my local test Hyper-V instance. Due to me skimping on resources, and making a typo, creating this instance was much harder than it should have been.

The first issue was I gave it a tiny data disk, this was down to me making a typo in my GB to Bytes conversion when specifying the size. Interestingly, the GHES setup does not initially complain but sits on the ‘reloading system services’ stage until it times out. If you check the /setup/config.log you see many Nomad related 500 errors. A reboot of the VM showed the real problem, the log then showed plenty of out-of-disk space messages.

reloading system devices does take a while

The easiest fix was to just start again with a data disk of a reasonable size

I next hit the problems due to my skimping on resources. I am not sure why I chose to limit them, old habits of using systems with scarce resources I guess.

I had only given the VM 10Gb of memory and 1 CPU. The Hyper-V host was not production-grade, but could certainly supply more than that.

  • The lack of at least 14Gb causes the GHES to fail to boot with a nice clear error message
  • The single CPU meant the ‘reloading application services’ step fails, the /setup/config.log shows the message
Task Group "treelights" (failed to place 1 allocation):
* Resources exhausted on 1 nodes
* Dimension "cpu" exhausted on 1 nodes

As soon as I stopped the VM and provided 14Gb of memory and multiple vCPU to the VM instance and rebooted the setup completed as expected.

So the top tip is to read the GHES systems requirements and actually follow them, even if it is just a test/demo instance.

Adding Job Summary support to my GitHub Release Notes Action

A recent addition to GitHub Actions is the ability to create a custom job summary. Using a simple echo command in a script you can write to the job summary

steps:   - name: Adding markdown
  run: echo '### Hello world! :rocket:' >> $GITHUB_STEP_SUMMARY

Now, this got me thinking. I have a well-used custom release notes extension for Azure DevOps. It has a GitHub Action equivalent, but it does not seem to get as much usage. Would adding support for custom Job Summaries make it more useful?

Well, there was only one way to find out, I added it.

There is now an extra boolean parameter for the action writeToJobSummary. If this is set to true (default is false) the output of the release notes action is written as a custom job summary (as well as still being available as a markdown file as before).

So now using this feature you see something similar to the following. The summary is built using a Handlbars based template.

I hope people find this a useful means to format their release notes using handlebars as opposed to having to build them using a series of echo statements in a script or writing their own custom action.

Fix for cannot ‘TypeError: Cannot read property’ when Dependabot submits a PR to upgrade a Jest Module

GitHub’s Dependabot is a great tool to help keep your dependencies up to date, and most of the time the PR it generates just merges without a problem. However, sometimes there are issues with other related dependencies.

This was the case with a recent PR to update jest-circus to 28.x. The PR failed with the error

TypeError: Cannot read property ‘enableGlobally’ of undefined at jestAdapter (node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:39:25) at TestScheduler.scheduleTests (node_modules/@jest/core/build/TestScheduler.js:333:13) at runJest (node_modules/@jest/core/build/runJest.js:404:19) at _run10000 (node_modules/@jest/core/build/cli/index.js:320:7) at runCLI (node_modules/@jest/core/build/cli/index.js:173:3)

In the end, the fix was simple, make sure all the other Jest related packages were updated to 28.x versions. Once I did this, using a GitHub Codespace, the PR merged without a problem.

Automating adding issues to Beta GitHub Projects using GitHub Actions

The new GitHub Issues Beta is a big step forward in project management over what was previously possible with the old ‘simple’ form of Issues. The Beta adds many great features such as:

  • Project Boards/Lists
  • Actionable Tasks
  • Custom Fields including Iterations
  • Automation

However, one thing that is not available out the box is a means to automatically add newly created issues to a project.

Looking at the automations available within a project you might initially think that there is a workflow to do this job, but no.

The ‘item added to project’ workflow triggers when the issue, or PR, is added to the project not when it is created. Now, this might change when custom workflows are available in the future, but not at present.

However, all is not lost. We can use GitHub Actions to do the job. In fact, the beta documentation even gives a sample to do just this job. But, I hit a problem.

The sample shows adding PRs to a project on their creation, but it assumes you are using GitHub Enterprise, as they make use of the ‘organization’ object to find the target project.

The problem is the ‘organization’ object was not available to me as I was using a GitHub Pro account (but it would be the same for anyone using free account).

So below is a reworked sample that adds issues to a project when no organization is available. Instead, making use of the ‘user’ object which also exposes the ProjectNext method

But what if I can’t use GitHub Codespaces? Welcome to github.dev

Yesterday GitHub released Codespaces as a commercial offering. A new feature I have been using during its beta phase.

Codespaces provides a means for developers to easily edit GitHub hosted repos in Visual Studio Code on a high-performance VM.

No longer does the new developer on the team have to spend ages getting their local device setup ‘just right’. They can, in a couple of clicks, provision a Codespace that is preconfigured for the exact needs of the project i.e the correct VM performance, the right VS Code extensions and the debug environment configured. All billed on a pay as you go basis and accessible from any client.

It could be a game-changer for many development scenarios.

However, there is one major issue. Codespaces, at launch, are only available on GitHub Teams or Enterprise subscriptions. They are not available on Individual accounts, as yet.

But all is not lost, hidden within the documentation, but widely tweeted about is the github.dev editor. You can think of this as Codespace Lite i.e. it is completely browser-based so there is no backing VM resource.

To use this feature, alter your URL https://github.com/myname/myrepo to https://github.dev/myname/myrepo . Or when browsing the repo just press the . (period) and you swap into a browser-hosted version of VS Code.

You can install a good number of extensions, just as long as they don’t require external compute resources.

So, this is a great tool for any quick edit that requires multiple files to be touch in the same commit.

I think it is going to be interesting to see how github.dev and Codespaces are used. Maybe we will see the end of massive developer PCs?

Or will that have to wait until the Codespace VM available offer GPUs?

A first look at the beta of GitHub Issue Forms

Update 10 May 2021 Remember that GitHub Issue Forms are in early beta, you need to keep an eye on the regular new releases as they come out. For example, my GitHub Forms stopped showing last week. This was due to me using now deprecate lines in the YAML definition files. Once I edited the files to update to support YAML they all leap back into life


GitHub Issues are core to tracking work in GitHub. Their flexibility is their biggest advantage and disadvantage. As a maintainer of a projects, I always need specific information when an issue is raised. Whether it be a bug, or feature request.

Historically, I have used Issue Templates, but these templates are not enforced. They add a suggestion for the issue text, but this can be ignored by the person raising the issue, and I can assure you they often do.

I have been lucky enough to have a look at GitHub Issue Forms, which is currently in early private beta. This new feature aims to address the problem by making the creation of issues form-based using YML templates.

I have swapped to using them on my most active repos Azure DevOps Pipeline extensions and GitHub Release Notes Action. My initial experience has been very good, the usual YML issue of incorrect indenting, but nothing more serious. They allow the easy creation of rich forms that are specific to the project.

They next step is to see if the quality of the logged issues improves.

Porting my Release Notes Azure DevOps Pipelines Extension to GitHub Actions

One of my most popular Azure DevOps Extensions is my Release Notes Pipeline task. This allows the creation of release notes using information obtained from the Azure DevOps API and formatted using a Handlebars Template.

Given the popularity of GitHub Actions, I got to wondering whether porting this extension was viable?

Well the release of my new Generate Release Notes with a Handlebars Template action shows that it was.

The basic concept of this new action is the same as for the older task, get information on the pipeline/workflow run using the API and then format it using a Handlebars template. However, the information that can be returned is different. But this stands to reason as GitHub is not Azure DevOps. This is especially true when you consider the differences between the simplicity of GitHub Issues and the complexity, and variability of format, of Azure DevOps Work Items

The new action is focused on the workflow run it is called from. It make an API call to get the details of the run. This contains a lot of information about the run and URL links to associated items. Using these link, the associated information is retrieved using the API and the results added to the objects available in the Handlebars template. In this initial version of the Action the objects tree available in a template includes:

  • runDetails – the details of the current workflow run
    • pull_requests – the array of pull requests associated with the run
      • commits – the array of commits associated with the PR
      • comments – the array of comment associated with the PR
      • linkedIssues – the array of linked issues with the PR

As with my Azure DevOps Extension I have made use of the extensibility of Handlebars. My new action includes all the Handlebar Helpers, plus some action specific helpers I have written and you have the ability to add your own custom handlebar helpers if needed.

So I hope people find this new Action useful, I guess only time will tell

Experiences setting up Azure Active Directory single sign-on (SSO) integration with GitHub Enterprise

Background

GitHub is a great system for individuals and OSS communities for both public and private project. However, corporate customers commonly want more control over their system than the standard GitHub offering. It is for this reason GitHub offers  GitHub Enterprise.

For most corporates, the essential feature that GitHub Enterprise offers is the use Single Sign On (SSO) i.e. allowing users to login to GitHub using their corporate directory accounts.

I wanted to see how easy this was to setup when you are using Azure Active Directory (AAD).

Luckily there is a step by step tutorial from Microsoft on how to set this up. Though, I would say that though detailed this tutorial has a strange structure in that it shows the default values not the correct values. Hence, the tutorial requires close reading, don’t just look at the pictures!

Even with close reading, I still hit a problem, all of my own making, as I went through this tutorial.

The Issue – a stray / in a URL

I entered all the AAD URLs and certs as instructed (or so I thought) by the tutorial into the Security page of GitHub Enterprise.

When I pressed the ‘Validate’ button in GitHub, to test the SSO settings, I got an error

‘The client has not listed any permissions for ‘AAD Graph’ in the requested permissions in the client’s application registration’

This sent me shown a rabbit hole looking at user permissions. That wasted a lot of time.

However, it turns out the issue was that I had a // in a URL when it should have been a  /. This was because I had made a cut and paste error when editing the tutorial’s sample URL and adding my organisation details.

Once I fixed this typo the validation worked, I was able to complete the setup and then I could to invite my AAD users to my GitHub Enterprise organisation.

Summary

So the summary is, if you follow the tutorial setting up SSO from AAD to GitHub Enterprise is easy enough to do, just be careful of over the detail.

Release of my video on ‘Introduction to GitHub Actions’

I recently posted on my initial experiences with GitHub Actions. I had hoped to deliver a session on this subject a DDD 14 in Reading , I even got so far as to propose a session.

However, life happened and I found I could not make it to the event. So I decided to do the next best thing and recorded a video of the session. I event went as far as to try to get the ‘DDD event feel’ by recording in front of a ‘live audience’ at Black Marble’s offices.

A first look at GitHub Action – converting my Azure DevOps tasks to GitHub Actions

Introduction

GitHub Actions open up an interesting new way to provide CI/CD automation for your GitHub projects, other than the historic options of Jenkins, Bamboo, Team City, Azure DevOps Pipelines etc. No longer do you have to leave the realm of GitHub to create a powerful CI/CD process or provision a separate system.

For people familiar with Azure DevOps YAML based Pipelines you will notice some common concepts in GitHub Actions. However, GitHub Action’s YAML syntax is different and Actions are not Tasks.You can’t just re-use your old Azure DevOps tasks.

So my mind quickly went to the question ‘how much work is involved to allow me to re-use my Azure DevOps Pipeline Tasks?’. I know I probably won’t be moving them to Docker, but surely I can reuse my Node based ones somehow?

The Migration Process

The first thing to consider is ‘are they of any use?’.

Any task that used the Azure DevOps API was going to need loads of work, if even relevant on GitHub. But, my Versioning tasks seemed good candidates. They still needed some edits, such as the logic to extract a version number from a the build number needed to be removed. This is because GitHub Actions have no concept of build numbers (it is recommended that versioning is done using SemVer and branching).

Given all this I picked one for migration, my JSONVersioner

The first step was to create a new empty GitHub repo for my new Action. I did this using the JavaScript template and followed the Getting Started instructions. This allowed me to make sure I had a working starting point.

I then copied in my JSON file versioner task into the repo bit by bit

  • Renamed my entry ApplyVersionToJSONFile.ts file to main.ts to keep inline with the template standard
  • Copied over the AppyVersionToJSONFuncitons.js
  • I removed any Azure DevOps specific code that was not needed.
  • In both files swapped the references to “vsts-task-lib/task” to “@actions/core” and update the related function calls to use
    • core.getInput()
    • core.debug()
    • core.warning()
    • core.SetFailed()
  • Altered my handling of input variables defaults to use the GitHub Actions as opposed to Azure DevOps Pipeline variables (to find the current working folder)
  • Migrated the manifest from the task.json
    to the action.yaml
  • Updated the readme.md with suitable usage details.

And that was basically it, the Action just worked, I could call my Action from a test workflow in another GitHub repo

However, I did decided to do a bit more work

  • I moved my Mocha/Chai based tests over to use Jest, again to keep inline with the template example. This was actually the main area of effort for me. Jest runs it’s tests async, and this caused me problem with my temporary file handling that had to be reworked. I also took the chance to improve the tests handing of the JSON comparison, making it more resilient for cross platform testing.
  • I also added TSLint to the npm build process, something I do for all my TypeScript based projects to keep up code quality

Summary

So the basic act of migration from Azure DevOps Pipeline Extension to GitHub Actions is not that hard if you take it step by step.

The difficultly will be with what your tasks do, are they even relevant to GitHub Actions? And are any APIs you need available?

So migration of Azure DevOps Extension Tasks to GitHub Actions is not an impossible task, have a look at my source at JSONFileVersioner or in the actual task in the GitHub Marketplace with the usage

jobs:

build:

runs-on: ubuntu-latest

strategy:

matrix:

node-version: [12.x]

steps:

     - uses: actions/checkout@v1

    - uses: rfennell/JSONFileVersioner@v1

with:

Path: ''

Field: 'version'

FilenamePattern: '.json'

Recursion: 'true'

    - name: Use Node.js ${{ matrix.node-version }}

uses: actions/setup-node@v1

with:

node-version: ${{ matrix.node-version }}

    - name: npm install, build, and test

run: |

npm install

npm run build --if-present

npm test

env:

CI: true




There is a nice series of posts on Actions from Microsoft’s Abel Wang – Github Actions 2.0 Is Here!!!