How to make jmeter to start many requests in a small time interval

Question

I want to make a load test on my API with 500k API calls.

I made a JMeter test that reads from a .csv file an access token and with that token make an API call.

Of course, from one computer I can't do that many requests so I implement distribute JMeter servers so I have on AWS one master and 500 servers that are the slave (t3.medium).

At the test, I set loops to be: 1000 (not so big number) Ramp Up: 5 seconds. I want the test to last for 5 seconds. So a total load of 500k API calls to be made during a 5 seconds frame. It doesn't matter when the test starts, but I need to make all the requests from all the slave servers during this 5 seconds interval.

And here is the problem that the test always takes longer. For example, it takes like 3 minutes to make all the requests, and the average response from the server is around 50ms.

How can I configure JMeter or what should I use in order to guarantee that all 500k API calls are made during a 5sec interval?

This is an example output from the master:

Starting remote engines
Starting the test @ Thu Nov 08 18:28:02 UTC 2018 (1541701682517)
summary +      1 in 00:00:00 =    4.8/s Avg:   106 Min:   106 Max:   106 Err:     0 (0.00%) Active: 76 Started: 24 Finished: 0
summary + 286890 in 00:00:27 = 10817.9/s Avg:    66 Min:    44 Max:  1363 Err:     0 (0.00%) Active: 2837 Started: 22851 Finished: 25632
summary = 286891 in 00:00:27 = 10732.9/s Avg:    66 Min:    44 Max:  1363 Err:     0 (0.00%)
summary +  40389 in 00:00:30 = 1344.8/s Avg:    67 Min:    45 Max:  1630 Err:     0 (0.00%) Active: 6607 Started: 27937 Finished: 24366
summary = 327280 in 00:00:57 = 5765.6/s Avg:    66 Min:    44 Max:  1630 Err:     0 (0.00%)
summary +  47986 in 00:00:30 = 1600.6/s Avg:    70 Min:    45 Max:  1618 Err:     0 (0.00%) Active: 2847 Started: 70360 Finished: 70549
summary = 375266 in 00:01:27 = 4326.1/s Avg:    67 Min:    44 Max:  1630 Err:     0 (0.00%)
summary +   8420 in 00:00:31 =  274.6/s Avg:    62 Min:    45 Max:   335 Err:     0 (0.00%) Active: 1231 Started: 77129 Finished: 78934
summary = 383686 in 00:01:57 = 3267.9/s Avg:    66 Min:    44 Max:  1630 Err:     0 (0.00%)
summary +   9173 in 00:00:30 =  305.1/s Avg:    59 Min:    45 Max:  1107 Err:     0 (0.00%) Active: 166 Started: 80976 Finished: 83846
summary = 392859 in 00:02:27 = 2663.8/s Avg:    66 Min:    44 Max:  1630 Err:     0 (0.00%)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 107131 in 00:01:11 = 1504.3/s Avg:    72 Min:    45 Max:  1230 Err:     0 (0.00%) Active: -484 Started: 86185 Finished: 88700
summary = 499990 in 00:03:39 = 2286.2/s Avg:    68 Min:    44 Max:  1630 Err:     0 (0.00%)
Tidying up remote @ Thu Nov 08 18:31:41 UTC 2018 (1541701901996)

And this is how JMeter test looks like

Dmitri T Dmitri T · Accepted Answer · 2018-11-09T06:39:25

There is one suitable option: Throughput Shaping Timer and Concurrency Thread Group combination. They can be put together via Schedule Feedback Function so JMeter will start extra threads if current amount is not enough to conduct the required load.

General recommendations:

Let's scale down a little bit, you need to ensure that 1 server can send 1000 requests in 5 seconds or 200 requests/second. It means that your server response time should be 5 milliseconds (including the time to establish the connection and requests to travel back and forth). As far as I can see minimum response time for your setup is 45 milliseconds and maximum is as high as 1,6 seconds. Therefore it looks like you will not be able to achieve the desired throughput using your current setup.

Now you need to identify the reason, here are some hints:

You have 5 seconds ramp-up, it means that last 200 users will be started on 5th second of your test, in any case it will take longer than 5 seconds
Amazon T3 medium is 2-core machine with 4 GB of RAM. Theoretically it should be enough to kick off 1000 users, however it depends on the size of request/response. Make sure JMeter engines are not overloaded using Amazon CloudWatch or JMeter PerfMon Plugin
Try running your test longer with a longer ramp-up time, it might give your application possibility to "warm-up" and you will be able to define the saturation point - the maximum number of users which your application can support without demonstrating performance degradation

How to make jmeter to start many requests in a small time interval

1 Answers