Jmeter 3.0: JSR223+Groovy vs. BeanShell

Question

Could you please help me to understand the difference between response times when i am using JSR223+Groovy (with caching) and BeanShell and what a reason of it:

The gray line is JSR223, red - BeanShell

Inside them a have a code which sending Protobuf message via HTTP protocol:

byte[] data = protobuf.toByteArray();
String SESSION = vars.get("SESSION");

String url = "http://" + vars.get("SERVER_NAME")+":8080/messages/list";
HttpClient client = new DefaultHttpClient();
////System.out.println(url);
HttpPost post = new HttpPost(url);
HttpEntity entity = new ByteArrayEntity(data);
post.setEntity(entity);
post.setHeader(HttpHeaders.CONTENT_TYPE, "application/x-protobuf");
post.setHeader(HttpHeaders.ACCEPT,"*/*");
post.setHeader(HttpHeaders.ACCEPT_ENCODING,"identity");
post.setHeader(HttpHeaders.CONNECTION,"Keep-Alive");
post.setHeader(HttpHeaders.USER_AGENT,"UnityPlayer/5.0.3p3 (http://unity3d.com) ");


post.setHeader("Cookie", "SESSION="+SESSION);
HttpResponse response=null;
try {
    response = client.execute(post);
} catch (ClientProtocolException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
response.getEntity().writeTo(baos);
byte[] b = baos.toByteArray();

Selva Selva · Accepted Answer · 2017-01-23T03:04:30

I observed the same issue with Jmeter v3.0 running on Sun Java 1.8. In my view, Groovy + JSR223 is memory intensive and loads tons of temporary objects and meta class objects. This increases the garbage collection overhead substantially. I was using G1GC as the GC algorithm.

Test #1 : Groovy + JSR223 - 8 requests / sec was the throughput achieved with 60 to 80% CPU and 3GB of heap space consumed

Test #2 : JSR223 + java (beanshell implementation) - GC overhead reduced considerably and the CPU % was somewhere around 40 to 60%. However, the throughput didn't improve. I observed plenty of thread locks for the bsh.ForName method

Test #3 : plain beanshell sampler with java code in it - This was the best! The throughput immediately reached an incredible 15000 per sec with CPU% at 100%. However, the overhead was due to the load. I reduced it down by 10% of the original load and it was able to achieved more than 50 req/sec easily with just 20% CPU.

Based on the experiement conducted above, I would suggest using plain beanshell sampler with java code in it instead of using JSR223.

Jmeter 3.0: JSR223+Groovy vs. BeanShell

1 Answers