The Issue

Whilst working on an Azure DevOps YAML pipeline for a solution that used Git Submodules we hit a problem with the checkout of the repo and submodule using the YAML

jobs:
  - job: 'Build'
    steps:
    - checkout: self
        submodules: true

It got the main Git repo, but failed with the following error

git submodule sync
git --config-env=http.https://myorg@dev.azure.com.extraheader=env_var_http.https://myorg@dev.azure.com.extraheader submodule update --init --force
Submodule 'Library' (https://myorg@dev.azure.com/myorg/myproject/_git/Library) registered for path 'Library'
Cloning into 'D:/a/1/s/Library'...
remote: TF401019: The Git repository with name or identifier Library does not exist or you do not have permissions for the operation you are attempting.

The Analysis

The issue is that the build agent access token was scoped to only the repo containing the YAML pipeline and not the submodule repo, even though they are in the same Azure DevOps Team Project.

This was not because of a lack of permissions for the build service account, but that the Azure DevOps Team Project pipelines setting ‘Protect access to repositories in YAML pipelines’ was enabled. This setting restricts the access of the build agent to only the repo containing the YAML pipeline, or ones referenced explicitly in the pipeline.

The Solution

The obvious solution is to disable the ‘Protect access to repositories in YAML pipelines’ setting for the Team Project. However, this may not be possible due to security requirements. In our case this setting was being enforced at the Team Project Collection level and we could not alter it.

But luckily there is a workaround.

1. In the YAML reference the Git Submodule and use the checkout task to clone into some folder (the folder is unimportant as we will never actually use it).

2. You can then call the main checkout with the submodules parameter set to true. This will now work because the agent access token is now scoped to both the repo the YAML is in and the submodule you manually referenced

name: $(Build.DefinitionName)
 
trigger:
  branches:
    include: [ main ] 
 
resources:
  repositories:
    - repository: Library # the submodule
      type: git
      name: Library
 
pool:
  vmImage: windows-latest
 
stages:
  - stage: 'Build_Packages'
    jobs:
      - job: 'Build'
        steps:
        - checkout: Library # the throwaway checkout
          path: './s/SomeTmpPath/Library'
 
        - checkout: self # the main checkout
          submodules: true

Note: You can’t seem to get away with just adding the reference, the explicit checkout of the referenced submodule repo is required, else you still get the permissions error.

So, a hacky solution, but it works!

For the original version of this post see Richard Fennell's personal blog at Checking out Git submodules when Azure DevOps Protected Access to repos is enabled