Integrate JMeter performance tests with Azure DevOps controlled by an Azure Pipeline

06-06-2021 door Roy de Kleijn

This blog post describes how to integrate JMeter performance tests with Azure DevOps. Microsoft comes with hosted agents, which you can use for free. At least you can play around for free, there are some limitations:

  • Public projects: each run can take max. 6 hours.

  • Private projects: each run can take max. 1 hour with a limitation of 30 hours a month.

You can use your own agents to bypass these limitations.

If you like to know more about the limitations, I like to suggest you read the following website: Capabilities and limitations.

Azure Pipeline Overview

You need to do a couple of things before your performance test is nicely integrated with Azure DevOps.
  1. First of all, the Microsoft hosted agents do not come with JMeter preinstalled. So, you need to take care of that.

  2. Then you want to override some of the default settings of JMeter, like the granularity.

  3. Finally, you want to implement clear reporting and downloadable artifacts.

Install JMeter
Add the following code snippet to install JMeter on the Microsoft Hosted Agent. You can specify a different version if you need to.
- script: |
wget -c
tar -xf apache-jmeter-5.4.1.tgz
displayName: 'Install the dependency packages'

Run script and override default settings
The following code snippet executes the JMeter script and overrides the default granularity setting.
- script: |
apache-jmeter-5.4.1/bin/./jmeter -Jjmeter.reportgenerator.overall_granularity=1000 -n -t test.jmx -l results/results.jtl -j results/output.log -e -o report
displayName: 'Run JMeter'

Generate JUnit report and downloadable artifacts
The following code snippet converts results.jtl to an JUnit output XML file. Test results will be displayed as shown in screenshot 3. The code snippet will also create downloadable artifacts from specific result directories.

- script: |
displayName: 'RESULTS: Convert JMeter Results to JUnit Format'

- task: PublishTestResults@2
testResultsFormat: 'JUnit'
testResultsFiles: 'output.xml'
failTaskOnFailedTests: true
displayName: 'RESULTS: Publish Load Testing Results'

- publish: $(System.DefaultWorkingDirectory)/results
artifact: jmeter-results
condition: succeededOrFailed()
displayName: 'RESULTS: Publish Load Test Artifacts'

- task: PublishPipelineArtifact@1
displayName: 'Publish JMeter HTML Report'
targetPath: report
artifact: jmeter-report

Azure DevOps Tests


Azure DevOps Tests / Filter

Code used in this repository is available on GitHub: Example JMeter Azure DevOps Integration