What does this messge java.net.ConnectException/Connection timed out mean in log.jtl file of Jmeter?

Question

I'd like to do a loading test with Jmeter. On google compute engine, I've creared 4 VMs, one running tomcat server, one running jmeter master and two running jmeter-server. My test plan is just a HTTP Request to the splash page(i.e. default home page, http://***.***.***.***/8080) of tomcat. There are 2000 users in each slave, and the plan will be executed in 5 minutes.

Message on the terminal of the jmeter master are as following:

Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary +      1 in 00:00:00 =    8.5/s Avg:     9 Min:     9 Max:     9 Err:     0 (0.00%) Active: 2 Started: 2 Finished: 0
summary + 300902 in 00:00:26 = 11425.9/s Avg:    75 Min:     0 Max:  1407 Err:     0 (0.00%) Active: 1745 Started: 1745 Finished: 0
summary = 300903 in 00:00:26 = 11375.0/s Avg:    75 Min:     0 Max:  1407 Err:     0 (0.00%)
summary + 321100 in 00:00:30 = 10703.3/s Avg:   243 Min:     0 Max:  7702 Err:     0 (0.00%) Active: 3727 Started: 3727 Finished: 0
summary = 622003 in 00:00:56 = 11018.1/s Avg:   162 Min:     0 Max:  7702 Err:     0 (0.00%)
summary + 314400 in 00:00:30 = 10478.6/s Avg:   372 Min:     0 Max: 31256 Err:     0 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 936403 in 00:01:26 = 10830.9/s Avg:   232 Min:     0 Max: 31256 Err:     0 (0.00%)
summary + 313700 in 00:00:30 = 10458.1/s Avg:   366 Min:     0 Max: 31827 Err:     0 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 1250103 in 00:01:56 = 10734.8/s Avg:   266 Min:     0 Max: 31827 Err:     0 (0.00%)
summary + 316200 in 00:00:30 = 10541.8/s Avg:   387 Min:     0 Max: 63556 Err:     0 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 1566303 in 00:02:26 = 10695.3/s Avg:   290 Min:     0 Max: 63556 Err:     0 (0.00%)
summary + 312100 in 00:00:30 = 10403.0/s Avg:   384 Min:     0 Max: 63661 Err:     0 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 1878403 in 00:02:56 = 10645.6/s Avg:   306 Min:     0 Max: 63661 Err:     0 (0.00%)
summary + 307900 in 00:00:30 = 10262.6/s Avg:   383 Min:     0 Max: 31917 Err:     0 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 2186303 in 00:03:26 = 10589.9/s Avg:   317 Min:     0 Max: 63661 Err:     0 (0.00%)
summary + 297500 in 00:00:30 = 9914.7/s Avg:   395 Min:     0 Max: 127296 Err:     2 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 2483803 in 00:03:56 = 10504.2/s Avg:   326 Min:     0 Max: 127296 Err:     2 (0.00%)
summary + 306000 in 00:00:30 = 10193.9/s Avg:   383 Min:     0 Max: 127265 Err:    20 (0.01%) Active: 4000 Started: 4000 Finished: 0
summary = 2789803 in 00:04:26 = 10469.3/s Avg:   332 Min:     0 Max: 127296 Err:    22 (0.00%)
summary + 314200 in 00:00:30 = 10483.1/s Avg:   389 Min:     0 Max: 127211 Err:     3 (0.00%) Active: 4000 Started: 4000 Finished: 0
summary = 3104003 in 00:04:56 = 10470.7/s Avg:   338 Min:     0 Max: 127296 Err:    25 (0.00%)
summary +  42788 in 00:00:29 = 1478.5/s Avg:   749 Min:     0 Max: 127326 Err:    43 (0.10%) Active: 0 Started: 4000 Finished: 4000
summary = 3146791 in 00:05:25 = 9670.9/s Avg:   344 Min:     0 Max: 127326 Err:    68 (0.00%)
Tidying up remote @ Wed Jan 11 03:26:58 UTC 2017 (1484105218946)

A few errors happened during the test. I looked at my log.jtl file and found they're ConnectException/Connection timed out.

Some messages are as following:

1484105011292,127254,HTTP Request,Non HTTP response code: java.net.ConnectException,Non HTTP response message: Connection timed out,jmeter-users 1-1708,text,false,,2111,0,2000,2000,0,0,127254
1484105011302,127244,HTTP Request,Non HTTP response code: java.net.ConnectException,Non HTTP response message: Connection timed out,jmeter-users 1-1897,text,false,,2111,0,2000,2000,0,0,127244
1484105138314,234,HTTP Request,200,,jmeter-users 1-80,text,true,,11369,183,2000,2000,234,0,0
1484105138279,269,HTTP Request,200,,jmeter-users 1-238,text,true,,11369,183,2000,2000,269,0,0
1484105011293,127253,HTTP Request,Non HTTP response code: java.net.ConnectException,Non HTTP response message: Connection timed out,jmeter-users 1-1565,text,false,,2111,0,2000,2000,0,0,127253
1484105138314,234,HTTP Request,200,,jmeter-users 1-1634,text,true,,11369,183,2000,2000,234,0,0
1484105138315,233,HTTP Request,200,,jmeter-users 1-1550,text,true,,11369,183,2000,2000,233,0,0
1484105138315,233,HTTP Request,200,,jmeter-users 1-1017,text,true,,11369,183,2000,2000,233,0,0

I've configured the timeout of tomcat and test plan, and set them to 300,000.
My question is why this error happened and how to configure to improve the throughput of tomcat.

EDIT(new error):
After adding a slave, I got a few following error in the log file(SocketException/Connection reset):

1484117548733,69717,HTTP Request,Non HTTP response code: java.net.SocketException,Non HTTP response message: Connection reset,jmeter-users 1-1749,text,false,,2629,0,34,34,0,0,118

Naveen Kumar R B Naveen Kumar R B · Accepted Answer · 2017-01-11T08:41:51

I think you are overloading the tomcat server.

ConnectException: Observed when all possible request processing threads (mapped to thread count & connection count) are busy serving the existing requests, then the requests will be kept in queue (of length configured in acceptCount parameter). If the requests did not get the chance, then timeout will occur and raise connectException.
SocketException: Observed when the queue (of acceptCount) is full, then subsequent requests will be immediately rejected.

As you are generating a load of 2000 vusers (on each slave), there must be supporting threads available on the server end also.

Tomcat configuration:

maxConnections: The maximum number of connections that the server will accept and process at any given time. When this number has been reached, the server will accept, but not process, one further connection
maxThreads: The maximum number of request processing threads to be created by this Connector, which therefore determines the maximum number of simultaneous requests that can be handled. If not specified, this attribute is set to 200.
acceptCount: The maximum queue length for incoming connection requests when all possible request processing threads are in use. Any requests received when the queue is full will be refused. The default value is 100.

These are important parameters to be considered while tweaking tomcat for better performance.

Note: Just increasing the numbers/threads won't result in better performance, as it completely depends on the hardware capabilities. so, it is like trail and error, where you need find the best combination with optimal performance that suits the given hardware.

What does this messge java.net.ConnectException/Connection timed out mean in log.jtl file of Jmeter?

1 Answers