We're migrating to SonarCloud because SonarQube's rule set has become outdated, lacking many of the latest checks. This move will ensure we're utilizing the most current standards for code quality and security directly in our GitHub workflow. The document will serve as your roadmap through the migration process. It’ll provide a comprehensive guide on configuring a GitHub workflow for code analysis with SonarCloud, applicable to both our Front-end and Back-end repositories.
Most of the repositories are already migrated. You can check it HERE.
IMPORTANT: To create or add new projects within the openLMIS organization on the SonarCloud platform, one must possess "Owner" rights. This requirement is crucial for disabling automatic analysis.
For demonstration purposes, we'll use the openlmis-requisition-ui repository to illustrate how to configure SonarCloud with GitHub Actions, providing a blueprint for enhancing code analysis in your front-end projects.
SonarCloud Account: Ensure you have a SonarCloud account connected to your GitHub.
Add a New Project:
Click on plus button near your avatar on the top right corner of the SonarCloud dashboard.
Select “Analyze new project”.
From the list of available repositories, select the repository that you want to analyze.
IMPORTANT: You do not have to worry about creating individual SonarCloud tokens for your project. They have been pre-configured at the organization level, making them available for all repositories under OpenLMIS.
Prior to setting up the configuration files, you need to adjust settings on the SonarCloud platform.
SonarCloud Project: Navigate to your project on the SonarCloud website.
Disable Automatic Analysis:
Click on Administration at the bottom-left corner of the page.
Select Analysis Method.
Disable Automatic Analysis which is Recommended.
Set Up Analysis with GitHub Actions:
After disabling automatic analysis, choose to set up analysis With GitHub Actions.
You will be presented with instructions specific to GitHub Actions – since the tokens are managed at the organization level, you can bypass the token setup.
Click on Other (for JS, TS, Go, Python, PHP, ...). Upon selecting this option, SonarCloud will provide specific instructions and snippets for setting up your repository. These instructions will be covered in the subsequent steps of this guide.
Once you've disabled Automatic Analysis on SonarCloud and chosen to set up analysis with GitHub Actions, the next step involves setting up your sonar-project.properties file. This file contains the configuration for the SonarCloud scan.
Project Root: Navigate to the root of the repository.
Properties File: Create a file named sonar-project.properties.
Configuration: Edit the file and add the basic configuration properties. Here’s an example of such a file for openlmis-requisition-ui:
sonar.projectKey=OpenLMIS_openlmis-requisition-ui sonar.organization=openlmis sonar.projectName=openlmis-requisition-ui sonar.sources=src sonar.sourceEncoding=UTF-8 |
IMPORTANT: While the provided example can serve as a starting point for your sonar-project.properties file, it's advisable to copy the configuration template directly from the SonarCloud platform and adjust it to match your project's specific needs.
The next crucial step is to define the GitHub Actions workflow that will run the SonarCloud analysis. This involves creating a workflow YAML file that tells GitHub Actions what to do.
Workflow Directory: If it doesn't already exist, create a directory named .github/workflows/ in the root of your repository.
Workflow File: Within the .github/workflows/ directory, create a new file named sonar-analysis.yml.
Workflow Content: Edit the sonar-analysis.yml file to include the steps that GitHub Actions will execute. Here’s an example workflow configuration used in openlmis-requisition-ui.
name: SonarCloud OpenLMIS-requisition-ui Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
sonarcloud:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: SonarCloud Scan
uses: SonarSource/sonarcloud-github-action@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
IMPORTANT: It's perfectly acceptable to use the provided example as a template for your workflow file. However, ensure you personalize the name of the workflow and other configurations to align with your project's specifics.
After the successful integration of SonarCloud with GitHub Actions, you can proceed with cleaning up legacy SonarQube configurations and scripts from your project repository. This helps to avoid confusion and ensures that the new workflow is the only source of truth for code analysis in your CI/CD pipeline.
Jenkinsfile Update:
Open the Jenkinsfile .
Locate the stage that is responsible for running SonarQube analysis.
Remove the entire stage that references SonarQube. This may look something like this:
stage('Sonar analysis) {
steps {
script {
// SonarQube related commands
}
}
} |
Delete SonarQube Scripts: If there is a sonar.sh script or any other scripts specifically designed to run SonarQube analysis, locate and delete them.
IMPORTANT: Before deleting any scripts, ensure they are not used by other stages or processes in your Jenkinsfile or other CI/CD scripts. Double-check that your new SonarCloud setup works as expected and that all team members are informed about the transition to avoid disrupting the workflow.
Here's a step-by-step guide to ensure that your backend repository is correctly set up for SonarCloud analysis with GitHub Actions. As previously, as a demo to illustrate how to integrate it properly, we’ll use the openlmis-requisition repository.
Just like with the Front-End repository, you'll need to set up your project on SonarCloud. Moreover, you have to disable Automatic Analysis.
SonarCloud Account: Ensure you have a SonarCloud account connected to your GitHub.
Add a New Project:
Click on Plus button near your avatar on the top right corner of the SonarCloud dashboard.
Select “Analyze new project”.
From the list of available repositories, select the repository that you want to analyze.
Enter newly created Project: Navigate to the main dashboard.
Disable Automatic Analysis:
Click on Administration at the bottom-left corner of the page.
Select Analysis Method.
Disable Automatic Analysis which is Recommended.
Set Up Analysis with GitHub Actions:
After disabling automatic analysis, choose to set up analysis With GitHub Actions.
You will be presented with instructions specific to GitHub Actions – since the tokens are managed at the organization level, you can bypass the token setup.
Click on Gradle | Groovy DSL. Upon selecting those options, SonarCloud will provide specific instructions and snippets for setting up your repository. These instructions will be covered in the subsequent steps of this guide.
Before proceeding with SonarCloud configuration, it's considered a best practice to ensure that your project builds successfully and all tests pass. This preliminary check can save time by identifying any existing issues that could interfere with the SonarCloud analysis.
Performing a Build Check: Run the following command in the root of your project to execute the build process and test execution.
docker-compose -f docker-compose.builder.yml run builder |
IMPORTANT: This step is not mandatory but is highly recommended. Verifying a successful build upfront can help streamline the SonarCloud integration process by ensuring that the codebase is in a stable state.
Once you’ve finished the basic configuration of your project, the next step involves workflow configuration. It is similar to the Front-End setup, but the script differs.
Workflow Directory: If it doesn't already exist, create a directory named .github/workflows/ in the root of your repository.
Workflow File: Within the .github/workflows/ directory, create a new file named sonar-analysis.yml.
Workflow Content: Edit the sonar-analysis.yml file to include the steps that GitHub Actions will execute. Here’s an example workflow configuration used in openlmis-requisition.
name: SonarCloud OpenLMIS-requisition Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
build:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: 17
distribution: 'zulu'
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- name: Build with Docker Compose
run: |
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env
docker-compose -f docker-compose.builder.yml run builder
sudo chown -R $(whoami) ./
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml
rm -rf ./build
- name: SonarCloud Scan
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: ./gradlew sonarqube --info
|
IMPORTANT: It's perfectly acceptable to use the provided example as a template for your workflow file. But keep in mind, that as previously, you have to personalize the name of the workflow and other configurations to align with your project's specifics.
The next step involves Gradle Wrapper check. The Gradle Wrapper is an important tool that ensures everyone uses the same Gradle version for your project, and it's crucial for running Gradle tasks within GitHub Actions, including the sonarqube task.
Identifying Gradle Version: If you have already performed a build check as described in step 1.5, you can determine the version of Gradle used by checking the contents of the .gradle directory within your project. Look for a folder named after the version number, which indicates the version of Gradle that the Wrapper is set to use.
Locating Gradle Wrapper Files: Confirm the presence of the following Gradle Wrapper files in your repository.
Check if gradlew and gradlew.bat scripts at the root of the project are available.
Check if gradle/wrapper/gradle-wrapper.jar and gradle/wrapper/gradle-wrapper.properties files within the gradle/wrapper directory are available.
IMPORTANT: If these files are present, your project is already configured to use the Gradle Wrapper. You can bypass next step.
Generating the Gradle Wrapper: If your repository does not include a Gradle Wrapper, you can generate one by following these steps.
Open a terminal and navigate to the root directory of your project.
Run the following command to generate the Gradle Wrapper:
gradle wrapper |
This command assumes you have Gradle installed locally. If not, install Gradle according to the official documentation before proceeding. As said before, you can check which version should be installed locally by checking the contents of the .gradle directory.
Once executed, the command will create the necessary files for the Gradle Wrapper in your project directory.
IMPORTANT: It's important to commit the Gradle Wrapper files (gradlew, gradlew.bat, gradle-wrapper.jar, and gradle-wrapper.properties) to your version control system. This inclusion ensures that all developers and the GitHub Actions workflow use the exact same Gradle build environment, which is essential for reliable SonarCloud analysis.
build.gradle for SonarCloudUpdate Plugin Versions: Ensure that all your plugins are updated to versions compatible with SonarCloud.
"org.sonarqube" : We need to upgrade the version to 3.3.
plugins {
id "org.sonarqube" version "3.3"
// ** The rest of plugins **
} |
"io.freefair.lombok": If such a plugin is available in plugins sections, you would have to delete it. Instead of it, you have to add some dependencies. Every instance of lombok should have a 1.18.22 version (this version ensures JDK 17 support).
dependencies {
compile "org.projectlombok:lombok:1.18.22"
annotationProcessor "org.projectlombok:lombok:1.18.22"
testAnnotationProcessor "org.projectlombok:lombok:1.18.22"
// ** The rest of dependencies **
} |
Configure SonarQube Task for SonarCloud: The sonarqube task in your build.gradle must be updated to target SonarCloud instead of SonarQube instance. Remove all lines related to a SonarQube server. This is how sonarqube task show look like after changes:
//NOTE: This plugin requires that this task be named 'sonarqube'. In fact, it is performing SonarCloud analysis.
sonarqube {
properties {
property "sonar.projectKey", "OpenLMIS_openlmis-requisition"
property "sonar.organization", "openlmis"
property "sonar.host.url", "https://sonarcloud.io"
property "sonar.java.source", "17"
property "sonar.coverage.jacoco.xmlReportPaths", "./report.xml"
}
} |
IMPORTANT: Some of properties could be copied from your project settings on SonarCloud platform. Moreover, it’s worth to add a comment that this task has to be named sonarqube even when we’re performing SonarCloud analysis. This is because our plugin version, it requires such a naming.
After you have successfully configured SonarCloud with GitHub Actions, it’s important to clean up any old references to SonarQube that might conflict with or duplicate the functionality of your new setup.
Jenkinsfile Update:
Open the Jenkinsfile .
Locate the stage that is responsible for running SonarQube analysis.
Remove the entire stage that references SonarQube. This may look something like this:
stage('Sonar analysis) {
steps {
script {
// SonarQube related commands
}
}
} |
Delete SonarQube Scripts: If there is a ci-sonarAnalysis.sh script or any other scripts specifically designed to run SonarQube analysis, locate and delete them.
IMPORTANT: Before deleting any scripts, ensure they are not used by other stages or processes in your Jenkinsfile or other CI/CD scripts. Double-check that your new SonarCloud setup works as expected and that all team members are informed about the transition to avoid disrupting the workflow.
During the migration process to SonarCloud using GitHub Actions, you may encounter scenarios where the build process is not proceeding as expected. These issues may be pre-existing from before the migration or arise due to the transition. Below are some edge cases and suggested adjustments to handle them.
If your build is failing for reasons unrelated to the SonarCloud migration, it's essential to resolve these issues. However, during the migration, you might decide to proceed with SonarCloud scans despite these failures. Such issues should be fixed as soon as possible but are beyond the migration's scope.
Here is the adjusted sonar-analysis.yml, which was used in openlmis-fulfillment, where build was unable to complete due to failures in the integration test suite.
name: SonarCloud OpenLMIS-fulfillment Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
build:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: 17
distribution: 'zulu'
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- name: Build with Docker Compose
continue-on-error: true
run: |
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env
docker-compose -f docker-compose.builder.yml run builder
sudo chown -R $(whoami) ./
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml
rm -rf ./build
- name: Analyze
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
sudo chown -R $(whoami) ./
./gradlew sonarqube --info |
Sometimes jacocoTestReport.xml might not be generated correctly. In these instances, we've established that SonarCloud scans should still proceed without the test coverage. This adjustment ensures that the migration can continue while issues with test coverage reporting are addressed separately.
Here's an example sonar-analysis.yml used in the openlmis-diagnostics repository, which was adjusted to continue with SonarCloud analysis despite the build process not producing a report:
name: SonarCloud OpenLMIS-diagnostics Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
build:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: 17
distribution: 'zulu'
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- name: Build with Docker Compose
continue-on-error: true
run: |
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env
sudo chown -R $(whoami) ./
docker-compose -f docker-compose.builder.yml run builder
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml
rm -rf ./build
- name: Analyze
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
sudo chown -R $(whoami) ./
./gradlew sonarqube --info |
When encountering failures with the ramlToHtml task , check the package.json file to ensure the runApiHtmlConverter script is present and that the paths are accurate. Incorrect paths can lead to this task failing. Here’s how the corrected script should look:
"scripts": {
"runApiHtmlConverter": "raml2html --input ./build/resources/main/api-definition-raml.yaml --output ./build/resources/main/api-definition.html"
}, |
IMPORTANT: Below is an example of the script with previously incorrect paths that could cause the ramlToHtml task to fail:
"scripts": {
"runApiHtmlConverter": "raml2html --input /app/build/resources/main/api-definition-raml.yaml --output /app/build/resources/main/api-definition.html"
}, |
Projects built using Gradle 6.4 may encounter issues when attempting to run with JDK 17 in the GitHub Actions workflow. This incompatibility requires the Java version used in the workflow to be downgraded to JDK 11 to ensure compatibility and successful execution of tasks. As a reference, I would use openlmis-template-service repository.
Here are the steps to resolve this issue:
Modify the sonar-analysis.yml file:
name: SonarCloud OpenLMIS-template-service Pipeline
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
build:
name: SonarCloud Analyze
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
java-version: 11
distribution: 'zulu'
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v3
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- name: Build with Docker Compose
continue-on-error: true
run: |
curl -o .env -L https://raw.githubusercontent.com/OpenLMIS/openlmis-ref-distro/master/settings-sample.env
docker-compose -f docker-compose.builder.yml run builder
sudo chown -R $(whoami) ./
cp ./build/reports/jacoco/test/jacocoTestReport.xml report.xml
rm -rf ./build
- name: Analyze
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
sudo chown -R $(whoami) ./
./gradlew sonarqube --info |
Update the "sonar.java.source" property in build.gradle:
sonarqube {
properties {
// ** The rest of properties
property "sonar.java.source", "11"
}
} |
IMPORTANT: After those changes, SonarCould Analysis would be performed. Nevertheless, it is advisable to consider fixing the incompatibility issue.