Jmeter - Distributed(Remote) Test Execution Slave Connection closed in middle of execution

0
votes

I am using Jmeter version 5.2.1. I have a scenario to simulate 400 parallel logins and inside the test plan I will execute the test samplers for 20 times using loop controller for each users(Threads).

Hence to fast track, I have executed it in remote test execution. Normally 200 Threads with 10 Loop count will take a maximum of 25 mins only. When increasing the loop count to 20, one of the slave node connection closed and the jenkins build went into dead loop.

I am using AWS M5.xLarge instance type. 4 CPU/16 GB RAM. Below is the run command I am using for Jmeter

JVM_ARGS="-Xms2g -Xmx4g -XX:MaxMetaspaceSize=256m"  && export JVM_ARGS && ./jmeter -n -t /home/jenkins/workspace/performance-test-staging-pipeline/test_plans/fd_regression_delight.jmx -e -o /home/jenkins/workspace/fd-performance-test-staging-pipeline/Performance-Report -l /home/jenkins/workspace/fd-performance-test-staging-pipeline/JTL-RPT-986.jtl -R xx.xx.xx.xxx,yy.yy.yy.yyy -Jserver.rmi.ssl.disable=true -Dclient.tries=3 -Gthreads=200

Below is the build log what I can see in Jenkins build console.

summary +   2700 in 00:00:59 =   45.4/s Avg:  3259 Min:    98 Max: 32684 Err:     0 (0.00%) Active: 400 Started: 400 Finished: 0
summary =  52102 in 00:25:38 =   33.9/s Avg:  6542 Min:    81 Max: 139355 Err:     0 (0.00%)

Below is the log I can see in jenkins-server.log

Connection to xx.x.x.xxx closed by remote host.
Connection to xx.x.x.xxx closed.
2
jmeter

2 Answers

0
votes

Looking into this line:

summary =  52102 in 00:25:38

it seems that JMeter executed 52102 requests in 25 minutes. I don't know how many requests does your test plan have, however I would expect something like:

  • for 1 request: 400 threads * 20 loops * 1 request = 8000
  • for 2 requests: 400 threads * 20 loops * 2 requests = 16000
  • for 3 requests: 24000 results
  • etc.

So make sure you provide correct number of loops or limit your test duration either on Thread Group level or using the Runtime Controller

Also when it comes to distributed testing every slave executes the test plan as it is so if you have 200 users set in the test plan and have 1 slave - you will have 200 users, for 2 slaves you will have 400 users, for 3 slaves - 600 users, etc. so if you need 200 users in total you might want to set -Gthreads=100 given you have 2 slave machines

Also pay attention to the response time, it seems that for certain samplers it exceeds 2 minutes, 2 minutes * 20 loops gives 40 minutes of total execution

0
votes

We are using spot instance to run our tests. Once I increase the draining timeout of the security group, my issue got resolved.