SPWakeUp (SPWakeUp3) v1.1.0 Released

I’ve implemented something that I’ve wanted to for a long time on SPWakeUp: The ability to wake additional URLs.

Version 1.1.0 allows the use of the ‘-Include’:’ command line parameter to specify additional URLs that will be woken once the detected site collections and subsites have been traversed.

SharePoint Online Content Type Hub Publishing Issues

We use the SharePoint Online Content Type Hub extensively as the central location to create, manage and distribute content types throughout our SharePoint Online tenant. We recently saw an issue with the distribution of one of the content types defined in the Content Type Hub which had a specific PowerPoint template assigned to it.

The content type in question had been updated with a revised template and republished, however when we tried to create a new presentation from the content type in SharePoint Online, the older template was still being served. I spent quite a long time testing a variety of PowerPoint template files of varying content and sizes with the content type (upload a new template to the content type in the Content Type Hub –> republish the content type –> wait for distribution to the site collections –> test; a process that can take some time when the content type is being distributed correctly) with no progress at all. It didn’t seem to matter what template file I uploaded to the content type, nothing seemed to be distributed. I even tried a blank Word template, and even that didn’t get distributed. Eventually I raised a Microsoft support case to help resolve the issue.

As the content type used managed metadata columns and I’d made a minor change to those, I tested with content types created using the same columns but a different template file and one using no managed metadata columns, but with what appeared to be the problematic template file uploaded. Only the example with the problematic template file was affected, but in the same way that we’d seen with the original content type, it didn’t matter what template file we uploaded afterwards, the content type and revised template were not distributed. The template we were using was obviously the issue, however it seemed to break distribution of any content type it was uploaded to from then on.

SharePoint content type templates are held in a folder structure in the root of site collections as follows:

_cts/<content type name>/<template file name>

This is true for both the Content Type Hub where the content types are distributed from, and the destination site collections.

Examining this location in the Content Type Hub (initially using PowerShell, but also using the SharePoint Client Browser), showed the following:

SPCB exploration of cts folder

As can be seen, all of the template files that have been uploaded to the content type were still present! Uploading a new template file does not appear therefore to delete the previous file that was associated with the content type. If one of the uploaded files causes issues distributing the content type, even though it may not be associated with the content type anymore, it continues to cause issues simply by being present in the folder for the content type. For example the following PowerShell retrieves the template filename and URL associated with the content type:

Get-PnPContentType -Identity “Black Marble Presentation” | select DocumentTemplate,DocumentTemplateUrl

In our case, this demonstrated that the template that was causing the issue was not associated with the content type, except by its existence in the folder for the content type in _cts.

The following PowerShell could therefore be used to remove the extraneous template files from the content type folder in the Content Type Hub site collection:

Remove-PnPFile –SiteRelativeUrl “_cts/Black Marble Presentation/<template file name>”

Using the above PowerShell I removed all of the unused template files and then republished the content type with a simple blank Word template initially to test whether the content type was distributed successfully. Once that was shown to be successful I uploaded a revised PowerPoint template file and republished the content type. The revised template was also successfully distributed.

SPWakeUp (SPWakeUp3)–Wake Up On-Premises SharePoint and WSS Instances

Since I started working with on-premises SharePoint instances, one of the solutions that I’ve used to wake up (pre-compile) the site collections and sub-sites contained within the web applications hosted by the farm is SPWakeUp.

This was originally a solution hosted on CodePlex and provided binaries for SharePoint 2007, then later SharePoint 2010 (the archive containing those can still be downloaded from the CodePlex Archive). I created compiled binaries for SharePoint 2013 and SharePoint 2016 and made those available as well.

I recently had need to use SPWakeUp on SharePoint 2019, so decided to produce a compiled for that version. As SPWakeUp doesn’t seem to have an active home anymore, I thought that it may be worthwhile putting the code and compiled versions on GitHub in case anyone else wants to use them! Note that if anyone objects to this happening, let me know and I’ll pull it down.

At the moment the repository hosts the original source code, the source code upgraded for use with Visual Studio 2019, compiled versions of SPWakeUp for SharePoint 2013, SharePoint 2016 and SharePoint 2019 and some instructions on how to compile the source yourself using Visual Studio Community 2019.

I hope it’s useful to someone!

Correcting Legacy Microsoft Teams Settings

I recently noticed that some of the Microsoft Teams that we had in use exhibited different behaviour to others that had been created more recently. ‘Recently’ (for varying definitions of ‘recent’) created Teams allowed users to edit and delete their messages, whereas a few older Teams did not. The older Teams were all created very early during our rollout of Teams, when Microsoft Teams itself was relatively new.

Checking the Teams messaging policies showed that the correct settings were in place to allow users to edit and delete their messages:

Messaging policy settings

It appeared that these settings were not however being applied for the legacy Teams. Checking in the individual Team settings showed that these were overriding the global settings applied:

Individual Team settings

Note also that in the above screen shot, the ‘Owners can delete all messages’ setting was also unchecked, although this turned out to be the only Team for which this was the case.

I could go through each Team individually and manually check the Team settings, however PowerShell allows us to perform the same actions quickly. The following PowerShell checks for the user Team settings being mis-configured and corrects the settings:

Connect-MicrosoftTeams


$teams = Get-Team | where {$_.AllowUserDeleteMessages -eq $False -OR $_.AllowUserEditMessages -eq $False}


foreach ($t in $teams) {
     Set-Team -DisplayName $t.DisplayName -GroupId $t.GroupId -AllowUserEditMessages $True -AllowUserDeleteMessages $True
}

For the other mis-configured setting mentioned, use ‘AllowOwnerDeleteMessages’ as the search criteria.

Using GitVersion when your default branch is not called ‘master’

The Black Live Matter movement has engendered many conversations, hopefully starting changes for the good. Often these changes involve the use of language. One such change has been the move to stop using the name master and switching to the name main for the trunk/default branch in Git repos. This change is moving apace driven by tools such as GitHub and Azure DevOps .

I have recently had need, for the first time since swapping my default branch name in new repos to main, to use Semantic Version and the GitVersion tool.

‘Out of the box’ I hit a problem. The current shipping version of GitVersion (5.3.2) by default makes the assumption that’s the trunk branch is called master. Hence, throws an exception if this branch cannot be found.

Looking at the project’s repo you can find PRs, tagged for a future release, that address this constraint. However, you don’t have to wait for a new version to ship to use this excellent tool in repos with other branch naming conventions.

The solution is to create an override file GitVersion.yml in the root of your repo with the following content to alter the Regex used to find branches. Note that the content below is as a minimum, you can override any other default configuration values in this file as needed.

branches:
master:
regex: ^master$|^main$

With this override file the default branch can be either master or main.

You can of course use a different name or limit the Regex to a single name as you need.

How do handle PRs for Azure DevOps YAML Pipelines if the YAML templates are in a different repo?

Azure DevOps YAML base pipelines allow the pipeline definitions to be treated like any other code. So you make changes in a branch and PR them into the main/trunk when they are approved.

This works well if all the YAML files are in the same repo, but not so well if you are using YAML templates and the templated YAML is stored in a different repo. This is because an Azure DevOps PR is limited to a single repo. So testing a change to a YAML template in a different repo needs a bit of thought.

Say for example you have a template called core.yml in a repo called YAMLTemplates and you make a change to it and start a PR. Unless you have a test YAML pipeline in that repo, which is not a stupid idea, but not always possible depending on the complexity of your build process, there is no way to test the change inside that repo.

The answer is to create a temporary branch in a repo that consumes the shared YAML template. In this temporary branch make an edit to the repository setting that references the shared YAML repo to point to the update branch contain the PR

resources: 
repositories:
  - repository: YAMLTemplates
   type: git
   name: 'Git Project/YAMLTemplates'
# defaults to ref: 'refs/heads/master'
ref: 'refs/heads/newbranch'

You don’t need to make any change to the line where the template is used

extends:  
template: core.yml@YAMLTemplates
  parameters:
    customer: ${{parameters.Customer}}
    useSonarQube: ${{parameters.useSonarQube}}

You can then use this updated pipeline to validated your PR. Once you are happy it works you can

  1. Complete the PR in the YAML Templates repo
  2. Delete the temporary branch in your consuming repo.

How can I automatically create Azure DevOps Release Notes and how can I publish them?

A question I am often asked when consulting on Azure DevOps is ‘how can I automatically create release notes and how can I publish them?’.

Well it is for just this requirement that I have written a set of Azure DevOps Pipeline Tasks

  • Release Note Generator – to generate release notes. I strongly recommend this Cross-platform Node-based version. I plan to deprecate my older PowerShell version in the not too distant future as it uses ‘homegrown logic’, as opposed to standard Azure DevOps API calls, to get associated items.
  • Wiki Updater – to upload a page tot a WIKI.
  • WIKI PDF Generator – to convert a generated page, or whole WIKI, to PDF format.

So lets deal with these tools in turn

Generating Release Notes

The Release Note task generates release notes by getting the items associated with a build (or release) from the Azure DevOps API and generating a document based on a Handlebars based template.

  • The artefacts that can be included in the release notes are details of the build/release and associated Work Items, Commits/Changesets, Tests and Pull Requests.
  • Most of the sample templates provided are for markdown format files. However, they could easily be converted for other text-based formats such as HTML if needed.
  • The use of Handlebars are the templating language makes for a very flexible and easily extensible means of document generation. There are sample of custom extensions provided with the templates

Sample YAML for this task is as follows, not it is using an inline template but it is possible to also load the template from a file path

        - task: richardfennellBM.BM-VSTS-XplatGenerateReleaseNotes.XplatGenerate-Release-Notes.XplatGenerateReleaseNotes@3
          displayName: 'Generate Release Notes'
          inputs:
            outputfile: '$(System.DefaultWorkingDirectory)inline.md'
            outputVariableName: OutputText
            templateLocation: InLine
            inlinetemplate: |
              # Notes for build 
              **Build Number**: {{buildDetails.id}}
              **Build Trigger PR Number**: {{lookup buildDetails.triggerInfo 'pr.number'}} 

              # Associated Pull Requests ({{pullRequests.length}})
              {{#forEach pullRequests}}
              {{#if isFirst}}### Associated Pull Requests (only shown if  PR) {{/if}}
              *  **PR {{this.id}}**  {{this.title}}
              {{/forEach}}

              # Builds with associated WI/CS ({{builds.length}})
              {{#forEach builds}}
              {{#if isFirst}}## Builds {{/if}}
              ##  Build {{this.build.buildNumber}}
              {{#forEach this.commits}}
              {{#if isFirst}}### Commits {{/if}}
              - CS {{this.id}}
              {{/forEach}}
              {{#forEach this.workitems}}
              {{#if isFirst}}### Workitems {{/if}}
              - WI {{this.id}}
              {{/forEach}} 
              {{/forEach}}

              # Global list of WI ({{workItems.length}})
              {{#forEach workItems}}
              {{#if isFirst}}## Associated Work Items (only shown if  WI) {{/if}}
              *  **{{this.id}}**  {{lookup this.fields 'System.Title'}}
                - **WIT** {{lookup this.fields 'System.WorkItemType'}} 
                - **Tags** {{lookup this.fields 'System.Tags'}}
              {{/forEach}}

              {{#forEach commits}}
              {{#if isFirst}}### Associated commits{{/if}}
              * ** ID{{this.id}}** 
                -  **Message:** {{this.message}}
                -  **Commited by:** {{this.author.displayName}} 
                -  **FileCount:** {{this.changes.length}} 
              {{#forEach this.changes}}
                    -  **File path (TFVC or TfsGit):** {{this.item.path}}  
                    -  **File filename (GitHub):** {{this.filename}}  
              {{/forEach}}
              {{/forEach}}

How to Publish The Notes

Once the document has been generated there is a need for a decision as to how to publish it. TThere are a few options

  • Attach the markdown file as an artefact to the Build or Pipeline. Note you can’t do this with a UI based Releases as they have no concept of artefacts, but this is becoming less of a concern as people move to multistage YAML.
  • Save in some other location e.g Azure Storage or if on-premises a UNC file share
  • Send the document as an email – I have used Rene van Osnabrugge Send Email Task for this job.
  • Upload it to a WIKI using my WIKI Updater Task
  • Convert the markdown release note document, or the whole WIKI, to a PDF and use any of the above options using first my WIKI PDF Exporter Task then another task.

I personally favour the 1st and 4th options used together. Attachment to the pipeline and then upload the document to a WIKI

A sample of suitable YAML is shown below, uploading the document to an Azure DevOps WIKI. Please note that the repo URL and authentication can trip you up here so have a good read of the provided documentation before you use this task.

  - task: richardfennellBM.BM-VSTS-WIKIUpdater-Tasks.WikiUpdaterTask.WikiUpdaterTask@1
          displayName: 'Git based WIKI Updater'
          inputs:
            repo: 'dev.azure.com/richardfennell/Git%20project/_git/Git-project.wiki'
            filename: 'xPlatReleaseNotes/build-Windows-handlebars.md'
            dataIsFile: true
            sourceFile: '$(System.DefaultWorkingDirectory)inline.md'
            message: 'Update from Build'
            gitname: builduser
            gitemail: 'build@demo'
            useAgentToken: true

But when do I generate the release notes?

I would suggest you always generate release notes every build/pipeline i.e. a document of the changes since the last successful build/pipeline of that build definition. This should be attached as an artefact.

However, this per build document will usually too granular for use as ‘true’ release notes i.e. something to hand to a QA team, auditor or client.

To address this second use case I suggest, within a multistage YAML pipeline (or a UI based release), having a stage specifically for generating release notes.

My task has a feature that it will check for the last successful release of a pipeline/release to the stage it is defined in, so will base the release note on the last successful release to that given stage. If this ‘documentation’ stage is only run when you are doing a ‘formal’ release, the release note generated will be since the last formal release. Exactly what a QA team or auditor or client might want.

In conclusion

So I hope that this post provides some ideas as to how you can use my tasks generate some useful release notes.

Fix For: ‘The pipeline is not valid error: Unable to resolve latest version’ on an Azure DevOps YAML pipeline

The Issue

I have an Azure DevOps multi-stage YAML pipeline that started giving the error `The pipeline is not valid error: Unable to resolve latest version for pipeline templates: this could be due to inaccessible pipeline or no version is available` and failing instantly.

image

The Solution

This is not the most helpful message, but after some digging I found the problem.

The pipeline used another pipeline as a resources

resources: 
pipelines:
- pipeline: templates
source: QueuesAndFunctionsDemo-CI
branch: master

This referenced build had failed, so there was no successful build resources to load, hence the error.

Once the problem with this reference build was fixed the error message went away and I could trigger my build

Exporting Release Notes and WIKIs as PDFs using a new Azure DevOps Extension that wrappers AzureDevOps.WikiPDFExport

A common question I get when people are using my Release Notes task for Azure DevOps is whether it is possible to get the release notes as a PDF.

In the past, the answer was that I did not know of any easy way. However, I have recently come across a command line tool by Max Melcher called AzureDevOps.WikiPDFExport that allows you to export a whole WIKI (or a single file) as a PDF. Its basic usage is

  • Clone a WIKI Repo
  • Run the command line tool passing in a path to the root of the cloned repo
  • The .order file is read
  • A PDF is generated

This is a nice and simple process, but it would be nice to be able to automate this process as part of a build pipeline.

After a bit of thought, I realised I had much of the code I needed to automated the process in my WIKIUpdater extension as these tasks are based around cloning repos.

So I am please to say I have just released a new Azure DevOps extension WikiPDFExport that wrappers Max’s command line tool. It does the following

  • Downloads the latest release of the WikiPDFExport tool from GitHub to the build agent (the exe is too big to include in the VSIX package)
  • Optionally clone a Git based WIKI repo. As with my WikIUpdater tasks, you can pass credentials for Azure DevOps or GitHub
  • Generate a PDF of a single file or the whole of a Wiki folder structure (based on the .order file) that was either cloned or was already present on the agent

A sample of the YAML usage of the task is as shown below. For full documentation see the extensions wiki pages for general usage and troubleshooting and the full YAML specification

- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1 
  displayName: 'Export Single File generated by the release notes task'
  inputs:
    cloneRepo: false
    localpath: '$(System.DefaultWorkingDirectory)'
    singleFile: 'inline.md'
    outputFile: '$(Build.ArtifactStagingDirectory)PDFsingleFile.pdf'
- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1
   displayName: 'Export a public GitHub WIKI'
   inputs:
     cloneRepo: true
     repo: 'https://github.com/rfennell/AzurePipelines.wiki.git' 
     useAgentToken: false
     localpath: '$(System.DefaultWorkingDirectory)GitHubRepo' 
     outputFile: '$(Build.ArtifactStagingDirectory)PDFpublicGitHub.pdf'
- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@1
   displayName: 'Export a private Azure DevOps WIKI'
   inputs:
     cloneRepo: true
     repo: 'https://dev.azure.com/richardfennell/GitHub/_git/GitHub.wiki' 
     useAgentToken: true
     localpath: '$(System.DefaultWorkingDirectory)AzRepo' 
     outputFile: '$(Build.ArtifactStagingDirectory)PDFAzrepo.pdf'

So hopefully this new extension will give teams another way to present their release notes, whether it be an export of a whole WIKI or just a single page.