Where did all my test results go?


I recently tripped myself up whist adding SonarQube analysis to a rather complex Azure DevOps build.

The build has two VsTest steps, both were using the same folder for their test result files. When the first VsTest task ran it created the expected .TRX and .COVERAGE files and then published its results to Azure DevOps, but when the second VsTest task ran it over wrote this folder, deleting the files already present, before it generated and published it results.

This meant that the build itself had all the test results published, but when SonarQube looked for the files for analysis only the second set of test were present, so its analysis was incorrect.


The solution was easy, use different folders for each set of test results.

This gave me a build, the key items are shown below, where one VsTest step does not overwrite the previous results before they can be processed by any 3rd party tasks such as SonarQube.

 2- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4  
 3   displayName: 'Prepare analysis on SonarQube'  
 4   inputs:  
 5     SonarQube: Sonarqube  
 6     projectKey: 'Services'  
 7     projectName: 'Services'  
 8     projectVersion: '$(major).$(minor)'  
 9     extraProperties: |  
10      # Additional properties that will be passed to the scanner,   
11      sonar.cs.vscoveragexml.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.coveragexml  
12      sonar.cs.vstest.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.trx
16… other build steps
20\- task: VSTest@2  
21   displayName: 'VsTest – Internal Services'  
22   inputs:  
23     testAssemblyVer2: |  
24      \*\*\*.unittests.dll  
25      !\*\*obj\*\*  
26     searchFolder: '$(System.DefaultWorkingDirectory)/src/Services'  
27     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsServices'  
28     overrideTestrunParameters: '-DeploymentEnabled false'  
29     codeCoverageEnabled: true  
30     testRunTitle: 'Services Unit Tests'  
31     diagnosticsEnabled: True  
32   continueOnError: true
34\- task: VSTest@2  
35   displayName: 'VsTest - External'  
36   inputs:  
37     testAssemblyVer2: |  
38      \*\*\*.unittests.dll  
39      !\*\*obj\*\*  
40     searchFolder: '$(System.DefaultWorkingDirectory)/src/ExternalServices'  
41     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsExternalServices'  
42     vsTestVersion: 15.0  
43     codeCoverageEnabled: true  
44     testRunTitle: 'External Services Unit Tests'  
45     diagnosticsEnabled: True  
46   continueOnError: true  
48\- task: BlackMarble.CodeCoverage-Format-Convertor-Private.CodeCoverageFormatConvertor.CodeCoverage-Format-Convertor@1  
49   displayName: 'CodeCoverage Format Convertor'  
50   inputs:  
51     ProjectDirectory: '$(System.DefaultWorkingDirectory)'  
53\- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4  
54   displayName: 'Run Code Analysis'
56\- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4  
57   displayName: 'Publish Quality Gate Result'