Jmeter HTTP Request Response Time too low compared to SoapUI

0
votes

We are using Apache JMeter (latest version) for load tests of a high performance API. JMeter response time is too high in comparison to SoapUI (free version). SoapUI gets an average response time of 200 milliseconds while JMeter reports at least 1200 milliseconds for the same API, testing from the same machine.

This affects the calculated throughput in each application. On SoapUI I can get up to 850 tps for said API, no errors. On JMeter I'm getting 190 tps at most, no matter what.

Tests are running with 200 threads, for 120 seconds.

I have already tweaked jvm/jmeter heap mmemory, garbage collector, and so on to no avail. Nothing changes. Can anyone help with this? Thank you.

Edit: Added test setup sample images

JMeter Thread Group and Test Plan Overall Setup

SoapUI TestSuite and Overall Setup

Edit 2: test results

Jmeter test results 1105 ms

SoapUi test results 263 ms

1
jmetersoapuiperformance-testing
Give example of your test plan,also are you testing web site or unrelated API calls? - user7294900
@user7294900 thank you for your reply Testing unrelated API calls. The test is simple: - 1 thread group, containing 1 HTTP request sampler. It is a post request that sends a small json payload to the API. I run the same test using 200 threads for 60 seconds on both JMeter and SoapUI and get totally different results in terms of throughput and response time. - JUP
*120 seconds, not 60 seconds - JUP
Are you sending payload inside Body data in HTTP request? - user7294900
Yes, a very small json payload . Object with a single value array. - JUP

1 Answers

1
votes

I can only think of one possible reason: you're running JMeter in GUI mode and this is not something you should be doing as JMeter isn't capable of sending requests fast enough, JMeter's GUI mode is designed for tests development and debugging.

Your images don't tell the full story as JMeter test results are missing. It would be also beneficial to generate a HTML Reporting Dashboard and see the correlation between increasing number of virtual users and response time/throughput.

And last but not the least, JMeter can be executed in Distributed Mode and in SoapUI it's available only for LoadUI Pro