Azure devops jmeter load test - how to access your jmeter summary reports

Question

Usually one would create one or more linux VMs, and run one or more jmeter master/slaves. Then you can collect the output of the threadgroups summary report listener, which contains fields like average, min, max, std.deviation, 95 percentile etc.

When you run your jmeter project in devops under "Load tests"->New->"Apache Jmeter Test", it does output some standard info under charts, summary and logs, but this is not the output from your summary report listener, it must be the output from some other report listener. It does have total average response time (not response time per api call which I need), and doesn't have std. deviation, 95th percentile etc. which I get when I run the project manually in jmeter myself. Under the devops jmeter tool it does have jmeter.logs and DefaultCTLAttributes.csv, but neither of these contain my summary data.

how do I get the devops jmeter tool to output my summary report listener?

Answer 1

Based on my test, I could reproduce this situation. The test result( jmeter.logs and DefaultCTLAttributes.csv) in Test Plan -> Load test indeed doesn't contain the fields(e.g. min,max,std.deviation).

It seems that there is no option to create summary that could contain these points.

For a Workaround, you could run the Jmeter test in the Pipeline.

For example:

steps:
- task: CmdLine@2
  inputs:
    script: |
      cd JmeterPath\apache-jmeter-5.3\apache-jmeter-5.3\bin
      jmeter -t Path\Jmeter.jmx -n -l Path\report.jtl

- task: CmdLine@2
  inputs:
    script: |
      cd  Jmeterpath\apache-jmeter-5.3\apache-jmeter-5.3\bin
      
      jmeter -g Path/report.jtl -o  OutPutPath

Since the hosted agents haven't install the Jmeter, you need to run the Pipeline on Self-hosted agents.

Then you could get the Chart in the Html file. This Html file contains these information.

If you want to publish this file to Azure Devops, you could use the Publish Build Artifacts task.

On the other hand, you can report your needs to our UserVoice website.

Hope this helps.

Answer 2

You also can use the extension called Taurus available on: https://marketplace.visualstudio.com/items?itemName=AlexandreGattiker.jmeter-tasks

Answer 3

To get the JMeter Reports available as an Azure DevOps Pipeline tab you can also use the extension https://marketplace.visualstudio.com/items?itemName=LakshayKaushik.PublishHTMLReports&targetId=c2bac9a7-71cb-49a9-84a5-acfb8db48105&utm_source=vstsproduct&utm_medium=ExtHubManageList , with htmlType='JMeter' .

The Post https://lakshaykaushik2506.medium.com/running-jmeter-load-tests-and-publishing-jmeter-report-within-azure-devops-547b4b986361 provides the details with a sample pipeline.

Answer 4

You can also use the following pipeline template: https://github.com/Azure-Samples/jmeter-aci-terraform

It leverages Azure Container Instances as JMeter agents. It publishes a JMeter dashboard (with those metrics you need) as a build artifact.