Out of Memory running SonarQube Analysis on a large projects

Whilst adding SonarQube analysis to a large project I started getting memory errors during the analysis phase. The solution was to up the memory available to the SonarQube Scanner on the my build agent, not the memory on the SonarQube server as I had first thought. This is done with an environment variable as per the documentation, but how best to do this within our Azure DevOps build systems?

The easiest way to set the environment variable `SONAR_SCANNER_OPTS` on every build agent is to just set it via a Azure Pipeline variable. This works because the build agent makes all pipeline variables available as environment variables at runtime.

So as I was using YML Pipeline, I set a variable within the build job

job: build
timeoutInMinutes: 240
variables:
- name: BuildConfiguration
value: 'Release'
- name: SONAR_SCANNER_OPTS
value: -Xmx4096m
steps:

I found I had to quadruple the memory allocated to the scanner. Once this was done my analysis completed

Running SonarQube for a .NET Core project in Azure DevOps YAML multi-stage pipelines

We have been looking migrating some of our common .NET Core libraries into new NuGet packages and have taken the chance to change our build process to use Azure DevOps Multi-stage Pipelines. Whilst doing this I hit a problem getting SonarQube analysis working, the documentation I found was a little confusing.

The Problem

As part of the YAML pipeline re-design we were moving away from building Visual Studio SLN solution files, and swapping to .NET Core command line for the build and testing of .CSproj files. Historically we had used the SonarQube Build Tasks that can be found in the Azure DevOps Marketplace to control SonarQube Analysis. However, if we used these tasks in the new YAML pipeline we quickly found that the SonarQube analysis failed saying it could find no projects

##[error]No analysable projects were found. SonarQube analysis will not be performed. Check the build summary report for details.

So I next swapped to using use the SonarScanner for .NET Core, assuming the issue was down to not using .NET Core commands. This gave YAML as follows,

- task: DotNetCoreCLI@2
     displayName: 'Install Sonarscanner'
     inputs:
       command: 'custom'
       custom: 'tool'
       arguments: 'install --global dotnet-sonarscanner --version 4.9.0
- task: DotNetCoreCLI@2
     displayName: 'Begin Sonarscanner'
     inputs:
       command: 'custom'
       custom: 'sonarscanner'
       arguments: 'begin /key:"$(SonarQubeProjectKey)" /name:"$(SonarQubeName)" /d:sonar.host.url="$(SonarQubeUrl)" /d:sonar.login="$(SonarQubeProjectAPIKey)" /version:$(Major).$(Minor)'

…. Build and test the project 

- task: DotNetCoreCLI@2
  displayName: 'En Sonarscanner'
  inputs:
    command: 'custom'
    custom: 'sonarscanner'
    arguments: 'end /key:"$(SonarQubeProjectKey)" '

However, this gave exactly the same problem.

The Solution

The solution it turns out was nothing to do with using the either of the ways to trigger SonarQube analysis, it was down to the fact that the .NET Core .CSProj file did not have a unique GUID. Historically this had not been an issue as if you trigger SonarQube analysis via a Visual Studio solution GUIDs are automatically injected. The move to building using the .NET core command line was the problem, but the fix was simple, just add a unique GUID to each CS project file.

<Project Sdk="MSBuild.Sdk.Extras">
  <PropertyGroup>
     <TargetFramework>netstandard2.0</TargetFramework>
     <PublishRepositoryUrl>true</PublishRepositoryUrl>
     <EmbedUntrackedSources>true</EmbedUntrackedSources>
     <ProjectGuid>e2bb4d3a-879c-4472-8ddc-94b2705abcde</ProjectGuid>

…

Once this was done, either way of running SonarQube worked.

After a bit of thought, I decided to stay with the same tasks I have used historically to trigger analysis. This was for a few reasons

  • I can use a central Service Connector to manage credentials to access SonarQube
  • The tasks manage the installation and update of the SonarQube tools on the agent
  • I need to pass less parameters about due to the use of the service connector
  • I can more easily include the SonarQube analysis report in the build

So my YAML now looks like this

- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B7A063157.SonarQubePrepare@4
  displayName: 'Prepare analysis on SonarQube'
  inputs:
    SonarQube: Sonarqube
    projectKey: '$(sonarqubeKey)'
    projectName: '$(sonarqubeName)'
    projectVersion: '$(Major).$(Minor)'
    extraProperties: |
          # Additional properties that will be passed to the scanner, 
          # Put one key=value per line, example:
          # sonar.exclusions=**/*.bin
          sonar.dependencyCheck.reportPath=$(Build.SourcesDirectory)/dependency-check-report.xml     
       sonar.dependencyCheck.htmlReportPath=$(Build.SourcesDirectory)/dependency-check-report.html
          sonar.cpd.exclusions=**/AssemblyInfo.cs,**/*.g.cs
             sonar.cs.vscoveragexml.reportsPaths=$(System.DefaultWorkingDirectory)/**/*.coveragexml
         sonar.cs.vstest.reportsPaths=$(System.DefaultWorkingDirectory)/**/*.trx

… Build & Test with DotNetCoreCLI

- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4
  displayName: 'Run Code Analysis'

- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4
  displayName: 'Publish Quality Gate Result'

Addendum..

Even though I don’t use it in the YAML, I still found a use for the .NET Core SonarScanner commands.

We use the Developer edition of SonarQube, this understands Git branches and PR. This edition has a requirement that you must perform an analysis on the master branch before any analysis on other branches can be done. This is because the branch analysis is measured relative to the quality of the master branch. I have found the easiest way to establish this baseline, even if it is of an empty project, is to run SonarScanner from the command on my PC, just to setup the base for any PR to be measured against.

Where did all my test results go?

Problem

I recently tripped myself up whist adding SonarQube analysis to a rather complex Azure DevOps build.

The build has two VsTest steps, both were using the same folder for their test result files. When the first VsTest task ran it created the expected .TRX and .COVERAGE files and then published its results to Azure DevOps, but when the second VsTest task ran it over wrote this folder, deleting the files already present, before it generated and published it results.

This meant that the build itself had all the test results published, but when SonarQube looked for the files for analysis only the second set of test were present, so its analysis was incorrect.

Solution

The solution was easy, use different folders for each set of test results.

This gave me a build, the key items are shown below, where one VsTest step does not overwrite the previous results before they can be processed by any 3rd party tasks such as SonarQube.

steps:
- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4
   displayName: 'Prepare analysis on SonarQube'
   inputs:
     SonarQube: Sonarqube
     projectKey: 'Services'
     projectName: 'Services'
     projectVersion: '$(major).$(minor)'
     extraProperties: |
      # Additional properties that will be passed to the scanner,
      sonar.cs.vscoveragexml.reportsPaths=$(System.DefaultWorkingDirectory)/**/*.coveragexml
      sonar.cs.vstest.reportsPaths=$(System.DefaultWorkingDirectory)/**/*.trx


… other build steps


- task: VSTest@2
   displayName: 'VsTest – Internal Services'
   inputs:
     testAssemblyVer2: |
      ***.unittests.dll
      !**obj**
     searchFolder: '$(System.DefaultWorkingDirectory)/src/Services'
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsServices'
     overrideTestrunParameters: '-DeploymentEnabled false'
     codeCoverageEnabled: true
     testRunTitle: 'Services Unit Tests'
     diagnosticsEnabled: True
   continueOnError: true

- task: VSTest@2
   displayName: 'VsTest - External'
   inputs:
     testAssemblyVer2: |
      ***.unittests.dll
      !**obj**
     searchFolder: '$(System.DefaultWorkingDirectory)/src/ExternalServices'
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsExternalServices'
     vsTestVersion: 15.0
     codeCoverageEnabled: true
     testRunTitle: 'External Services Unit Tests'
     diagnosticsEnabled: True
   continueOnError: true

- task: BlackMarble.CodeCoverage-Format-Convertor-Private.CodeCoverageFormatConvertor.CodeCoverage-Format-Convertor@1
   displayName: 'CodeCoverage Format Convertor'
   inputs:
     ProjectDirectory: '$(System.DefaultWorkingDirectory)'

- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4
   displayName: 'Run Code Analysis'

- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4
   displayName: 'Publish Quality Gate Result'