It's the SonarQube Elasticsearch indexes again

calendarJun 28, 2024 · 2 min read  ·
Share on: linkedincopy

Background

I just upgraded our Azure/Docker container hosted SonarQube instance from 10.5.1 to 10.6.0. This was partly due to our usual upgrade process, we try to upgrade within a week or so of a new release, but also to address a specific Java issue we started to see when using the SonarQube@6 Azure DevOps tasks.

The error was the SonarQube analysis completed successfully, but the clean up process failed with this JVM error.

INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 4:20.150s
INFO: Final Memory: 55M/194M
INFO: ------------------------------------------------------------------------
##[error]ERROR: Cleanup during JVM shutdown failed
ERROR: Cleanup during JVM shutdown failed
##[error]java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError

The Issue

The upgraded appeared to go well

  • The container restarted
  • The DB schema was updated
  • The SonarQube instance started and became available.
  • As expected, a background re-index started on our 89 projects

88 were re-indexed without issue, but one failed with the following error.

Error Details: A Customer Function [Project Data Reload]
Error Details
java.lang.IllegalStateException: Unrecoverable indexing failures: 7 errors among 7 requests. Check Elasticsearch logs for further details.

The Elasticsearch logs showed nothing more.

Solution

A search of the ever useful SonarSource Community forums suggested this error can be seen when there is a lack of disk space. However given we are running on Azure Storage and are using a tiny fraction of our available 100Tb of disk space, that did not seem to be the likely issue.

So I adopted my usual approach for SonarQube issues involving Elasticsearch. I stopped the container, deleted the index sonarqube-data/es6 folder, restarted the container and let SonarQube rebuild the index to see of this fixed the problem.

An as is so often the case, this fixed the issue, all the projects were re-indexed successfully.