API Perfromance testing tool - JMeter or JUnit?

Question

i am working on some performance testing task. The main goal is to compare duration of old NCP protocol calls againts new REST API calls. I have this scenario:

1. Client has an authenticated session
2. Client access protected resource

I have to create two variants:

a) One-by-one variant: The question is: How long does it take to perform 2000 requests sent one by one?

b) Concurrent variant: The question is: How long does it take to perform 2000 Request which are sent concurrently (300 Threads ideal)

I dont know the best way to solve this problem. My idea is:

a) Creation of 2000 Http clients -> Each client sends HTTP Post with credentials in body -> Each client sends HTTP GET and get the response (I will measure the time between sending the GET request and getting a response for each iteration and Sum it.

b) Creation of 2000 Httpclients -> Use executor service with fixed thread pool (300) -> each thread will perform sending get request.

Is there any other way? I know that Jmeter is a great tool but i am not sure that this scenario could by performed on Jmeter. Thanks!

Answer 1

For the second variant: you need to determine what is you targeted throughput (TP). 2000 request per hour? Per minute? Per second? Once you get the TP, and a guesstimate for the scenario response time (RT), you could estimate the number of VUsers using the Little's Law. Alternatively, you can use a calculator to determine that number.

Jmeter provides a mechanism to submit this workload (scenarios) by using Arrivals Thread Group. This TG will instantiate the number of threads needed to sustain the targeted TP.

Be aware that there is possibility that you might not reach the TP goal due to:

1. the SUT does not have the capacity to handle the load
2. a bottleneck (resource saturation) somewhere in the environment
3. the client (JMeter) does have enough resources to produce the load

Answer 2

JUnit itself doesn't provide any multithreading logic, you will have to construct the HTTP requests yourself (or with a 3rd-party library like RestAssured) and then execute them using i.e. ExecutorService or jmh and then come up with something for results analysis.

JMeter has everything out of the box so you won't need to write a single line of code, reporting is also included, it might be not that CI friendly as JMeter .jmx scripts are XML but on the other hand you will get nice protocol metrics and ability to correlate increasing load with increasing response time